The Central Intelligence Agency said this week that it will post its database of declassified CIA documents online, making them broadly accessible to all interested users.
The database, known as CREST (for CIA Records Search Tool), contains more than 11 million pages of historical Agency records that have already been declassified and approved for public release.
Currently, however, CREST can only be accessed through computer terminals at the National Archives in College Park, MD. This geographic restriction on availability has been a source of frustration and bafflement to researchers ever since the digital collection was established in 2000. (See CIA's CREST Leaves Cavity in Public Domain, Secrecy News, April 6, 2009; Inside the CIA's (Sort of) Secret Document Stash, Mother Jones, April 3, 2009).
But that is finally going to change.
The entire contents of the CREST system will be transferred to the CIA website, said CIA spokesperson Ryan Trapani on Tuesday.
"When loaded on the website they will be full-text searchable and have the same features currently available on the CREST system at NARA," he said.
CIA was not able to provide a date for completion of the transfer, but "we are moving out on the plan to make the transition," Mr. Trapani said.
In the meantime, "The CREST database housed at NARA will remain up and running at least until the website is fully functioning," he said.
Michael Best from Glomar Disclosure had been running a crowd-funded FOIA program to get them to print out each document. This was apparently costing the CIA money, so it helped tick the box for approval of the documents to finally be released.
An additional note -
Quote:Notes Glomar Disclosure - "I'm pleased to say Wikileaks is interested in partnering to make the pending release of 775,000+ CIA documents (13,000,000 pages) as searchable and accessible as possible."
Here are some further details of what is coming. There is some obvious material of interest here, including numerous previously unreleased CIA reports and materials from the first five Directors of Central Intelligence, including Dulles and Helms.
Quote:Releasing the records
A bit less than a year ago, I embarked on a quest to get a copy of the millions of pages of CIA documents stored on CREST, the CIA Records Search Tool. The CREST database was technically publicly available, in the sense that anyone could theoretically use the four computers located in the back of a library that (for budgetary reasons) lacks a librarian for half of the day. These four computers are currently the only ones that can access the CREST database, and they're only accessible Monday through Friday from 9 Am to 4:30 PM. In other words, most people who aren't full time researchers can't use the database even if they're within driving distance. By printing out and scanning the documents at CIA expense, I was able to begin making them freely available to the public and to give the Agency a financial incentive to simply put the database online. I'm pleased to say that these efforts have been a success, and the Agency is putting the database online.
CIA isn't doing this out of the goodness of their hearts. Several FOIA requests have been filed for the database, including by the National Security Archive and MuckRock. MuckRock actually sued the Agency with the help of Kel McClanahan of National Security Counselors. The Agency said it would take 28 years to process the files. After some more legal pressure from Mr. McClanahan, the Agency reduced their estimate to six years. This was still too long, and so I began my effort. The hope was that the financial pressure, the negative press and making it not only a legal but a practical inevitability that these files would be put online would force the Agency to speed up their timetable. Thanks to the combined (but uncoordinated) efforts of myself and MuckRock, these files will soon be available.
However, there are some problems with the Agency's statement about the matter. According to the Agency, the database will retain all of its features when put online. This is extremely unlikely, since the database is currently interfaced through proprietary software known as Laserfiche. This allows for many browsing and search parameters and sorting functions that the Agency's website simply doesn't. In many instances, the Agency's website is simply broken and users are unable to properly browse the categories of documents already uploaded. It's more likely that the Agency spokesman was unaware of this and only meant to refer to the text recognition that has been performed on the files. Otherwise, the Agency will have to design or buy an entirely new interface.
To combat this, I'm preparing to reindex and reupload the files in a proper format. While the CREST system at the National Archives has been out of toner for several weeks (CIA has been extremely and deliberately slow in this regard), more toner is expected to arrive this week. This will allow me to retrieve copies of the indexes with the metadata. This will, in turn, be used to organize the files and upload them in a fully searchable format. Assuming the Agency doesn't retain all of the search functions of the Laserfiche powered system and simply imports the files into their already broken interface, a new database will be built from the files. Several options are being considered in this regard and more than one organization has expressed interest in partnering over it.
What's in it?
So what's in the database? There are a little over 775,000 files that make up over 13,000,000 pages. Before the most recent update of files at the beginning of the year, the database was estimated to be about 840 gigabytes. Breaking these files down into categories, we get:
Secretary of State Henry Kissinger's papers: 40,000 pages of newly declassified documents. The papers did not originate with CIA, but "contain many CIA equities."
Directorate of Science and Technology R&D: 20,000 pages
Analytic intelligence publication files: Over 100,000 pages.
News archives: The Agency collected a lot of news stories about themselves and the subjects they were interested in. Their news archive, much of which is included in CREST, contains many topics of interest.
Office of the DCI Collection (ODCI): 28,550 documents/129,000 pages from the records of the first five Directors of Central Intelligence: Admiral Roscoe Hillenkoetter, General Walter "Bedell" Smith, Allen Dulles, John McCone, and Richard Helms. These records run from the beginning of CIA in 1947 through the late 1960s and include a wide variety of memos, letters, minutes of meetings, chronologies and related files from the Office of the DCI (ODCI) that document the high level workings of the CIA during its early years.
Directorate of Intelligence (DI) Central Intelligence Bulletins: 8,800 documents/123,000 pages from a collection of daily Central Intelligence Bulletins (CIB), National Intelligence Bulletins (NIB) and National Intelligence Dailies (NID) running from 1951 through 1979. The CIBs/NIBs were published six days a week (Monday through Saturday) and were all source compilations of articles and consisting initially of short Daily Briefs and longer Significant Intelligence Reports and Estimates on key events and tops of the day. The CIBs/ NIBs were circulated to high level policy-makers in the US Government.
General CIA Records: Records from the CIA's archives that are 25 years old or older, including a wide variety of finished intelligence reports, field information reports, high-level Agency policy papers and memoranda, and other documents produced by the CIA.
STAR GATE: A 25-year Intelligence Community effort that used remote viewers who claimed to use clairvoyance, precognition, or telepathy to acquire and describe information about targets that were blocked from ordinary perception. The records include documentation of remote viewing sessions, training, internal memoranda, foreign assessments, and program reviews.
Consolidated Translations: Translated reports of foreign-language technical articles of intelligence interest, organized by author and each document covers a single subject.
Scientific Abstracts: Abstracts of foreign scientific and technical journal articles from around the world.
Ground Photo Caption Cards: Used to identify photographs in the NlMA ground photograph collection. Each caption card contains a serial number that corresponds to the identical serial number on a ground photograph. The master negatives of the ground photography collection have been accessioned separately to NARA. The caption cards provide descriptive information to help identify which master negatives researchers may wish to request.
National Intelligence Survey: National Intelligence Survey gazetteers.
NGA: Records from the National Geospatial-Intelligence Agency, primarily photographic intelligence reports.
Joint Publication Research Service: Provided translations of regional and topical issues in the late 1970s and early 1980s.
Office of Strategic Services files: Documents from the OSS, CIA's World War II predecessor.
While these documents are older, they aren't irrelevant. One of the CREST documents provided the smoking gun for my expose on an NSA Director sabotaging the NSA.
The timeline on these files is a little bit sketchy at this point. It's unlikely that the files will be online before the election. If the Agency is going to keep their word and put the database online, it's likely that it'll happen in the next few months. New files are usually added to CREST between January and March, depending on the speed of bureaucracy. It's unlikely that they'll add these files to the offline database just before migrating them and hundreds of thousands of other files to an online database.
Work on reconstructing the database with all of the metadata can begin as early as next week. Once the metadata has been and organized work on the database itself can begin. 23,500 CIA documents (340,000 pages) have already been obtained and can be added to the database ASAP. The details of the timeline, however, depend on both when the Agency puts the files online and when potential partners are ready to move forward.
The article https://glomardisclosure.com/2016/10...nsa-promotion/ is very interesting and re-confirms my worst fears about the intelligence and military communities. Not only are they at war with the American population at large, but with themselves for turf and funding et al. They spy on us and on each other, on Presidents and their staff - on everyone and everything. Power corrupts and secret power corrupts even more.
I'd be very interested to be able to access CREST online, sure that it will track exactly who is looking at what, though.
"Let me issue and control a nation's money and I care not who writes the laws. - Mayer Rothschild
"Civil disobedience is not our problem. Our problem is civil obedience! People are obedient in the face of poverty, starvation, stupidity, war, and cruelty. Our problem is that grand thieves are running the country. That's our problem!" - Howard Zinn
"If there is no struggle there is no progress. Power concedes nothing without a demand. It never did and never will" - Frederick Douglass
According to this letter from the AARB to President Clinton, ALL documents related to Kennedy's assassination is now released, and according to a letter I received by the CIA who informed me they have now turned over all their documents to the AARB.
If there are more document to be release, then these documents must be prior to 1963 involving America's wars.
03-11-2016, 06:29 AM (This post was last modified: 03-11-2016, 07:07 AM by Peter Lemkin.)
Scott Kaiser Wrote:According to this letter from the AARB to President Clinton, ALL documents related to Kennedy's assassination is now released, and according to a letter I received by the CIA who informed me they have now turned over all their documents to the AARB.
If there are more document to be release, then these documents must be prior to 1963 involving America's wars.
Sorry Scott, that is not true. There are many many many documents never released or said to be 'unable to be found' or said to have 'been routinely destroyed'. A partial [!] list of as yet unreleased JFK Assassination documents is attached here. According to the JFK Act, all documents that are 'releasable' [a very contentious matter in and of itself] are to be released by next year [2017] - by law. However, many groups are fighting tooth and nail to see this happens, with the expectation that the really 'good' ones that show who really was behind it will NOT be released - as if they were, US polity would collapse and everyone would see that they had been manipulated and lied to not only in Dallas, but forever and completely. CAPA is working on seeing that a maximum of documents be released next year, as is AARB and many other groups. The list of other document groups that are said to be 'unable to be found' or said to have 'been routinely destroyed' is long, and I don't have the time just now to enumerate them. Some may have been destroyed, but more likely they went into the 'do not file file', so that the various agencies and entities could say they 'don't exist' [in the normal records].
N.B. I can not upload the list, I don't know why I can't upload it...but I have it on my computer!
Here is an article about the October 2017 release deadline. Most JFK researchers expect only a partial release (with the usual redactions!) of documents we know about and have not yet been released - how many others we do NOT know about is likely over a million. LHO's entire ONI file is 'missing'...as are many other important items.
With either Clinton or even less with Trump do I personally feel there will be any push from that 'branch' of the government to see that the JFK Act is carried out in deed, as it is written in word.
Quote:Exciting news: WhoWhatWhy has obtained the complete list of 3,603 secret documentson the Kennedy assassination still being held by the US government. (Or, to be precise, what it admits to still holding.) Now we can at least get a peek at what they have been hiding. The list was obtained in a Freedom of Information Act request by FOIA specialist Michael Ravnitzky, who alerted us. The complete list is below. You'll note that some documents are briefly characterized by subject, while others are less clearly identified. The government has promised to release as many documents as possible in October, 2017, the 25th anniversary of the JFK Records Act, in which Congress mandated that all efforts be made to release everything in Washington's possession unless an overriding case can be made for withholding in the national interest. Some perhaps most of these documents could be released at that time. Then again, they may be further withheld. The CIA in particular is likely to argue that some are just too sensitive to be made public. Still, knowing their subject matter makes it easier to press for disclosure, and to hold the government accountable by insisting it justify any continued withholdings. Those who wish to look at the list should be forewarned that it's a bit like looking at hieroglyphics. Most of the names and brief references will mean something only to a very few. Among the documents that caught our eye: Lee Harvey Oswald's CIA "201 File." 201 files contain personality assessments Records on David Morales, David Atlee Phillips, E. Howard Hunt, William King Harvey and others considered by top researchers prime suspects for participation in the planning and implementation of Kennedy's murder. Documents on New Orleans oddball David Ferrie, District Attorney Jim Garrison, Jackie Kennedy, members of the Warren Commission staff, Jack Ruby, various anti-Castro Cubans and much much more. Tax returns of Michael Paine, who with his wife Ruth provided housing and more for the Oswalds Ruth also got Lee a job in the Texas School Book Depository from which he purportedly shot Kennedy; Michael worked for the defense contractor Bell Helicopter. Click here to view the full document (it's a long one!)
"Let me issue and control a nation's money and I care not who writes the laws. - Mayer Rothschild
"Civil disobedience is not our problem. Our problem is civil obedience! People are obedient in the face of poverty, starvation, stupidity, war, and cruelty. Our problem is that grand thieves are running the country. That's our problem!" - Howard Zinn
"If there is no struggle there is no progress. Power concedes nothing without a demand. It never did and never will" - Frederick Douglass
If any documents exposed US government also including any three letter agencies involvment I seriously doubt the United States would collapse. How? Could you give me an excample of how the US would collapse other then NOW knowing on paper what we've known all along.
And, if the really important documents the world is waiting on and fighting for are lost, missing, destroyed or otherwise will not be released then what's the point? SMH...
There's heaps of stuff still under lock and key. Hopefully we'll see a lot of it next year.
For some reason, it's really hard to know who's telling the truth, who's being honest, or, at the very least, who wants to be honest.
Sometime ago when some information was up at that site whowhat or why about the release of these documents I was the first person to call that information out, the next thing I know is that the author and or editor made a mistake, if that's what everyone wants to keep calling it.
Frankly, I'm tried of these mistakes everyone seems to be gett'en away with, and other folks who really tell the truth, they get crucified.
If you ask me there are way to many good people gett'en taken for, soon, someone is going to have to fight for the naive and expose these so called reseachers for who they really are, and I know what excatly Ed Kaiser would do to help those who cannot help themselves.
