IG: NOAA, NASA Launched Next-Gen Satellite With Known Issues, Scrubbed Performance Metrics from Contract

aapsky/Shutterstock

The main instrument of the GOES-R next-generation satellite constellation wasn’t working before launch but officials sent it into space anyway.

Persistent problems with the premier sensors of the GOES-R series satellites—designed to provide the next generation of weather observation for North America—were identified before launch and not properly tested or resolved, according to a new inspector general report.

Further, the Commerce Department IG found evidence that program managers changed the evaluation criteria for the contractor after the issues were identified—metrics that would have led to a 40-75% reduction in payment had they remained.

The $11 billion GOES-R series of satellites includes GOES-16—launched November 2016—and GOES-17—launched March 2018—as well as the pending GOES-T and GOES-U still in production. The satellite constellation is equipped with a set of next-generation sensors to better predict weather patterns, including the Advanced Baseline Imager, or ABI, the “most essential instrument for mission success of the GOES-R satellites,” according to the IG.

However, shortly after GOES-17 entered orbit, the cooling system for the ABI instrument malfunctioned, “severely degrading” the amount of data the satellite could collect, NOAA officials said at the time.

“This is a serious problem,” Steve Volz, administrator of the National Oceanic and Atmospheric Administration’s National Environmental Satellite, Data and Information Service, or NESDIS, said during a May 2018 briefing with reporters. “This is the premier Earth-pointing instrument on the GOES platform and the 16 channels … are important elements of our observing requirements.”

Engineers with NOAA and NASA worked to get the instrument largely operational, though IG auditors found the program office should have seen these issues coming.

Testing on the ABI in March 2017 prior to launch showed issues with the cooling system. Engineers devised a workaround but only tested the changes at “ambient temperatures and pressures,” according to the IG, which did not simulate the conditions in which the satellite would be operating in space.

With the modifications tested at ground level, the risk management team labeled the issue as fixed and “did not categorize the thermal anomaly as a failure,” the IG wrote.

Using NASA’s “Test as You Fly—Fly as You Test” protocol, the program “accepted the risk of not re-performing the thermal vacuum testing of the satellite and proceeded to launch. We found this level of verification for ABI to have been insufficient given its importance to the [key performance parameter] capability for the mission,” the report states. “Re-testing the modified instrument and satellite in thermal vacuum could have provided a higher confidence level in the verification of the modification or potentially additional actions to preclude failure on orbit.”

The ABI cooling system subsequently failed in April 2018, shortly after the satellite launched in March.

Engineers have since been able to get the GOES-17 satellite to deliver 97% of the data it was designed to, though it will never be able to operate at the level intended. The 16 ABI channels will be available during the two “cold seasons”—May through June and November through January—but only seven will be available 24-hours during the remaining months. The other nine channels will be out of commission for two to six hours each day during that time, as direct sunlight will make the system too hot to function.

As these issues first came to light in 2017, an IG investigation found the program office altered the contract deliverables in favor of the vendor, Florida-based Harris, a tech firm with numerous government contracts. 

Evaluation metrics included in the original contract with Harris would have “classified instrument performance as either ‘degraded,’ meriting a reduction of up to 40% of award fees or ‘severely degraded,’ which merits up to 75% reduction,” the IG wrote.

Instead, the contract language was alerted “to align with the expected performance inputs from the funding agency,” i.e., NOAA, according to a NASA memo. The program office told the IG these changes were meant to streamline the evaluation process, as other metrics covered “the instruments’ radiometric performance.”

“However, our examination of the changes made to the ABI contract … found that criteria related to the thermal system and infrared calibration target were the only ones removed from the degraded and severely degraded classifications, soon after the program became aware of thermal and calibration issues with the ABIs,” the IG wrote. “We found that these issues would have likely reduced the contractor’s award fees for on-orbit performance without this revision.”

Auditors said they were not able to “fully explore this matter,” and left it to senior management to investigate further.

"On the issue of contractor award fees, before the GOES-17 launch, NOAA and NASA initiated changes to the contract evaluation terms to align the on-orbit performance criteria with the utility of the data from a user perspective. These changes were made for all instruments, not just ABI. These changes will allow the government to more effectively assess the earned fee for on-orbit performance based on user needs," a NOAA spokesperson told Nextgov. "It’s important to note the on-orbit evaluation for the GOES-17 ABI has not yet occurred and will take into account the impact of the LHP issue and the overall performance of the ABI."

And while the GOES-16 and -17 satellites are technically operational at this point, the systems are not working as intended and will “affect the reliability of the instruments, satellites and, ultimately, the constellation availability,” the IG wrote.

Those technical and contracting issues will need to be resolved as NOAA and NASA continue work on the second half of the GOES-R program—GOES-T and GOES-U, which will be given number designations once in orbit.

While engineers have been focused on the main sensor—the ABI—a secondary instrument, the magnetometer, which monitors the Earth’s magnetic field, is also not meeting expectations. The IG audit found the GOES-R magnetometers are less accurate than technology deployed on older satellites.

“GOES-16’s magnetometer has been less accurate than magnetometers on the previous generation satellites—GOES-13, -14, -15—and GOES-17,” auditors found. “GOES-17’s magnetometer has also not met its accuracy requirements,” they added.

As these tools fail to meet expected capabilities, the Space Weather Prediction Center told the IG it “continues to rely on the previous generation of satellites for these measurements.”

The audit also found issues in how the program planned to position the satellites within the greater constellation—though the team was able to adapt with minimal issues. The investigation also discovered the lifecycle costs will likely be higher than the original $11.7 billion projected, as ground system infrastructure—servers, software and security—will have to be updated over the next three years.

Editor's Note: This article was updated with a statement from NOAA.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.