Outrageous: NOAA demands $262,000 fee for looking at their ‘public’ data
Anthony Watts / 1 hour ago March 27, 2015
Eric Worrall writes: It looks like NOAA have found a new way to stifle FOIA inquiries from the public. According to Steve Goddard, NOAA have just demanded a $262,000 administrative fee for zipping up a few raw data files.
-----------------------
I think most would agree this is totally over the top pricing. A few old data files a few programmes - a few seconds work.
BUT
What had they asked for?
Well this is from the NOAA response:
The National Environmental Information Center (which includes the former National Climatic Data Center) has access to a subset of the requested records. Very few if any letters, phone logs, memos, and other communications on this subject would be available. Historical internal and external emails are archived, though they are expensive to access and analyze due to unsupported technology.
As data stewardship – including homogeneity adjustments – has been central to NCDC’s mission for decades, determining which records are responsive to this extremely broad request will require significant resources. Responding to this part of the request would require retrieving, reviewing, and packaging many tens of thousands of items in at least 29 years of communications, if they can be located.
For investigating availability of items and code, some from obsolescent systems, evaluating content, and screening for non-responsive, deliberative, or personally identifying information, I estimate the following Federal-equivalent labor classes, durations and costs. The actual personnel, and sources of labor (Federal or contracted), would depend on conditions and workforce options available at the specific project start time.
So seems like more than a few data files.
From Goddards blog it seems that this is what is asked for:
kentclizbe says:
JN,
Please provide the complete source code used for processing raw US temperature data.
This message is to confirm your request submission to the FOIAonline application: View Request. Request information is as follows:
• Tracking Number: DOC-NOAA-2014-001602
• Requester Name: Kent Clizbe
• Date Submitted: 09/07/2014
• Request Status: Submitted
• Description: 1. Temperature Data Record Adjustments: Rationale, Methodology, Discussions–USG employees and others
For the NOAA/National Climactic Data Center: Please provide Internal and external e-mails, letters, phone logs, memos, and other communications, from, to, and between: government employees, external consultants, experts, advisors, or other parties regarding the rationale, methodology, and other issues concerning adjustments/homogenization or other changes to both the US and global temperature record data, from the beginning of the adjustments through today.
2. Temperature Data Computer Code Used to Process/Adjust/Homogenize US Temperature DataPlease provide the complete source code used for processing raw US temperature data.
Everything except the toilet paper again . see http://climateandstuff.blogspot.co.uk/2011/06/schnare-vs-uva-impossible-request.html for something similar.
Do they think that this is easy to obtain?
You cannot just zip emails and send since there may be personal information included.
You have to employ people to read and approve and redact addresses.
Then of course there is the problem of changing technology - from NOAA comments email backups are available but not in a format currently readable. - so find the hardware - write an interface - sort through the myriad of emails that dot do not contain private comms. This is not simple search and zip as Goddard suggests.
As an example WUWT carried an article about retrieving photos from 1960s NASA missions:
A message from Dennis Wingo:
The Lunar Orbiter Image Recovery Project (LOIRP), is a NASA ESMD funded project to recover the original Lunar Orbiter analog data which was recorded on an Ampex FR-900 2″ video instrumentation recorder. As far as we know, we have the last surviving drives of this type in the world. We have retired Ampex engineers working with us on this project but the FR-900 was a limited use machine (exclusively the U.S. government at the FAA, USAF, NASA).
What we need is to find any possible source of documentation (we know about the Stanford Archive and have been there many times) for the FR-900 or the possibility of actual machines...
This is what one section of NOAA were using in the 80s/90s.
Would they be backing up things like emails when data storage was so expensive? Would they be transferring irrelevant data like emails to new storage media?
Would they still have those tape drives/optical drives (not CDs but 12" disks)?
Date System Installed: NOAA's satellite data processing and storage system became operational in September 1990.
System Installed by: Integrator/Vendor.
NOAA's data storage system components include:
- Digital Equipment Corporation (DEC) VAX 8530 computer system with 96 MB memory and floating point accelerator, 4 VUP (speed).
- VAX 11-785 computer system with 64 MB of memory and floating point accelerator, 1.7 VUP (speed).
- VAX 6000-510 computer system with 128 MB of memory and floating point accelerator, 13 VUP (speed).
- Data communications network including Star Coupler configuration and Ethernet (Ethernet Server-16 ports); router to Internet, NASA, DAMUS, and NOAA backbone.
- User workstations consisting of 48 PCs.
- Multiple banks of magnetic hard disks, 9-track magnetic tape (800, 1600, 6250 bpi) drives, square tape (240 MB), and tape cartridge drives (5.2 GB and 2.3 GB).
- Sony optical disk jukebox; 50 disk capacity (50 x 6.0 GB disks).
- Technical system documentation, UPS power supply, and on-site systems maintenance staff.