Palantir Technologies: Crystal Ball For The Information Age

DEAR FRIENDS. IF YOU LIKE THIS ARTICLE AND WANT TO RECEIVE MORE COVERAGE ON THIS TOPIC, SUPPORT SOUTHFRONT WORK:

PayPal: southfront@internet.ru

Donation alerts: https://donationalerts.com/r/southfront

Gumroad: https://gumroad.com/southfront

Or via: http://southfront.org/donate/ or via: https://www.patreon.com/southfront,

BTC: 3Gbs4rjcVUtQd8p3CiFUCxPLZwRqurezRZ,

BCH ABC: qpf2cphc5dkuclkqur7lhj2yuqq9pk3hmukle77vhq,

ETH: 0x9f4cda013e354b8fc285bf4b9a60460cee7f7ea9

Illustrative image

Written by Robert Barlow exclusively for SouthFront

In Tolkien’s The Lord of the Rings novel, the palantíri are magical artifacts created ages before the events of books took place. The indestructible crystals could be used by a skilled user for communication or to see events of the past or future. However, misused palantiri can provide unreliable information with subsequent incorrect conclusions. Despite Gandalf’s cautions, leaders on both sides of the war made critical strategic errors based on visions from their palantír.

In 2020 on Earth, Palantir is a technology company that specializes in integrating databases, analyzing that data, and drawing actionable conclusions. Applications of their products range from monitoring how people and companies comply with administrative regulations to chains supply and tracking of terrorist activity.

Palantir CEO Alex Karp YouTube/Screenshot

Gotham, launched in 2008, is Palantir’s first product to be sold to government agencies and private companies.

It was designed to integrate and analyze large data sets with a special attention to fraud investigation, information warfare monitoring and counter-terrorism analysis.

In the last two decades, computer and data gathering capabilities have grown exponentially. While collecting raw data is relatively easy, searching and analyzing that data into meaningful information is much more difficult. There is also the problem of use case: software for an airline to analyze plane schedules will probably not be useful to a law enforcement agency tracking human trafficking activity. For that reason Palantir Technologies Inc was created in 2003. Founded by Peter Thiel, PayPal co-founder, the company received venture capital from private groups as well as the CIA investment arm, In-Q-Tel, and have recently went public with a direct listing on the NYSE (New York Stock Exchange). Gotham was created as a customizable toolset for users to implement according to their specific use case.

The system is designed to integrate data from multiple sources and run on the customers’ own servers so it can be tailored to their needs. Data can be entered once, e.g. a spreadsheet or a report, or through the application programming interface (API) to enter data from another system automatically. For example, if you need to create a program that catalogs headlines of newspaper articles with summary information and time/date, this would create a database or spreadsheet of information. Your implementation of Gotham could then use an API call to copy the data into its own database to add it to the data set. Gotham is capable of using a variety of industry standard formats and languages, such as Java, REST, XML, and JSON, so it can quickly and seamlessly integrate with existing databases and systems. This can be done automatically for any period of time and from any number of sources.

After collecting your data, Gotham allows you to search and connect data points. Using proprietary analytic technologies, the system connects simple data, such as matching a phone number in multiple records, and maps its relation to specific objects. People, places, things, and events can be connected via this automatic analysis, but connections can also be made by users interacting with the system. The interconnections can be represented in various graphs, maps, by object, or in custom views. For example, a pharmaceutical company might plot out a graph using information from internal databases and government statistics to adjust manufacturing schedules.

Changes to the data can be tracked with Gotham’s Revisioning Database. (Note: this does not appear to be connected to Revenants CIE LLC RevDB. RevDB was designed to track how, when, and by whom each data point was entered and every connection was made. An analyst can go back to a “snapshot” of data at any time to see what data and connections were in the database at a given moment; in traditional databases this is not possible. By tracking this metadata information, policy based, granular user security and compartmentalizing of information is also possible. This is especially helpful to the United States’ so-called “intelligence fusion centers” which consist of local and national law enforcement and intel agencies. All actions within Gotham databases are tracked in audit logs, which can also be configured with tamper-evident properties. The logging and revision properties of Gotham are also very useful in low-bandwidth and high-latency situations such as ships at sea and field collection teams.

Click to see full-size image

Modules add flexibility to Gotham:

Ava is a suite of artificial intelligence algorithms that can alert users to known data patterns when they are spotted. The module runs around the clock and is customizable.
The Table module is designed to run on “massive-scale” data to allow non-specialized users to analyze efficiently.
Users may collaborate and share information in real time with the Dossier module.
The Operations module “lets users collaborate live to plan, execute, and report on operations.” (LINK) Data is projected on a geographic map with the ability to add any data from Gotham to the map manually. A smartphone app even allows users to file and upload reports from the field for near real-time information.
Finally, the Nexus Peering module enables multiple instances of your data and analyses to be synchronized. Changes are detected and merged across sites automatically with two-way, partial or full data sharing support.

In the modern age of computing, most software is sold under the “Software as a Service” model. This means that updates and changes are made by the developers on a continual basis and then released to customers over the Internet as patches. Updates to the operating system and apps on your phone and computer are a good example of SaaS in the real world. However, for US government agencies, any information classified as “Secret” or above must be kept on computer systems designated and certified to run that level of classification. The information security policy supposes their complete separation from the public Internet. As mentioned above, Gotham was designed to run on the customers in-house servers without an Internet connection, which means Gotham was built from the ground up to serve the needs of agencies with classified information systems. Updates can be received from Palantir through encrypted channels over the Internet, certified and audited by agency information security experts, then installed on the Secret or Top Secret Gotham database. This allows non-Internet connected Gotham implementations to be updated to the latest versions for bug fixes, security, and new features.

All this adds up to a robust, largely automated database and analysis tool for large organizations. Gotham’s wide ranging capabilities and malleability made it a superpower in data analysis technology. Palantir’s latest update to Gotham, dubbed Titan, was released in 2019 with new data views and modules.

Gotham has a twin product, Foundry. Foundry seems to be identical in function, but it was built as a SaaS “cloud-native” counterpart to Gotham. According to Palantir’s website, all commercial customers had been migrated to Foundry by September 2020.

A specific use of Palantir’s technology by military and intelligence agencies is kept in secret. In the press, one can find dozens of generalized anecdotes on its use by IED hunters in Afghanistan, FBI agents tracking terrorists in the US, and bankers searching for fraud and profit. But individual uses and tactics are classified, which means the general public will probably not directly see the results for 50 years, if ever. Palantir’s blog provides some insight into non-profit organizations use.

A GCHQ diagram indicates how Palantir could be used as part of a computer network attack.

Palantir’s customers.
In a Securities and Exchange Commission (SEC) filing, Palantir declared: “We have chosen sides. Our software is used by the United States and its allies in Europe and around the world. Some companies work with the United States as well its adversaries. We do not. We believe that our government and commercial customers value this clarity.” (LINK) According to USAspending.gov, Palantir’s largest customer is the US Department of Defense, followed by Department of Homeland Security, Department of the Treasury, Department of Justice, and the SEC. Other noted agencies include Immigration and Customs Enforcement and the Internal Revenue Service. Leaked documents from 2013 show additional agencies to be NSA, FBI, CDC, US Military Academy West Point, the Joint IED-defeat organization and Allies, the Recovery Accountability and Transparency Board, and the National Center for Missing and Exploited Children. This confirms the wide range of applications, as well as the fact that the U.S. national security and financial agencies largely rely on Palantir’s products. As we’ll see later in this report, this has also expanded to health agencies.

Besides the USA, Palantir is especially popular with the European consumers. Its products are widely used in the states structures. In 2016, French internal intelligence service DGSI signed a 10 million euros contract with Palatir Technology that was only a drop of water in American group’s total turnover. Besides the French intelligence services, among the clients of Palantir are the Danish national police, state police in Hesse and North Rhine-Westphalia, Germany and, possibly, Dutch police. Some states were pushed to use Palantir software by COVID pandemic. While Germany, France, and Austria chose to develop their corona tracing apps by their own forces, the UK has invited Palantir into its health care system. In the UK Palantir was engaged to work alongside Amazon Web Services, Google and Microsoft to construct a data analytics program for NHSX’s work on Covid-19. Moreover, it was awarded oversight of the UK’s post-Brexit border and customs data.

The list of Palantir’s clients consists of such companies as Fiat Chrysler, pharmaceutical laboratories Sanofi and Merck, bank Credit Suisse and the Ferrari Formula 1 team. European multinational aerospace corporation Airbus is closely cooperating with Palantir. The Skywise Company was set up by Airbus with Palantir to create its predictive maintenance service offering.

Peter Thiel, co-founder and chairman of Palantir Technologies Inc., speaks during a news conference in Tokyo, Japan, on Monday, Nov. 18, 2019. The billionaire entrepreneur was in Japan to unveil a $150 million, 50-50 joint venture with local financial services firm Sompo Holdings Inc. Palantir Technologies Japan Co. will target government and public sector customers, emphasizing health and cybersecurity initially. Photographer: Kiyoshi Ota/Bloomberg via Getty Images

Reports of the use of Gotham by Europol had a wide public response.

European Commission commented on this issue:

“Europol has been using Palantir software for analyzing operational data as part of Task Force ‘Fraternité’ in 2016 and since mid-2017 for the operational analysis of all counter terrorism related data.

Customised Palantir Gotham software is used at Europol as a tool for operational analysis, in particular for the visualization of data sets and to identify new lines of investigation in support of the competent authorities in EU Member States and beyond. The product holds operational personal data in the context of counter-terrorism investigations and operations, processed in line with Europol’s applicable legal framework.”

Europol has had an indirect working relationship with Palantir since 2012 as sub-contracted through Dutch IT consulting company Capgemini. According to official statement by the European Commission, the maximum value of financial expenditure under the framework contract was EUR 7.5 million. The total value of the expenditure paid under the contract, including the respective services provided by the sub-contractor, was around EUR 4 million.

Among the other European bodies, there are the European aviation agency EASA, various police forces in EU member states that are using or have shown interest in “Gotham”. Palantir software is used by international organizations like the International Atomic Energy Agency.

The popularity of Palantir products in the structures of the United States and “its allies” demonstrates the inability of European countries to ensure the security of national or regional structures. The direct link between Palantir and American intelligence unequivocally confirms Washington’s high level of access to any data available in European countries.

Palantir has a growing list of customers and a parabolic stock price increase since their direct listing on 30SEP2020. The company was valued at $10.5 billion at the time of the listing, but since then, the stock price has increased by 300%. Financial prognosticators have differing opinions on the future of the company’s dollar value and growth potential.

Illustrative Image

It is also worth mentioning Palantir’s controversies.

In March 2018, a whistleblower testified before United Kingdom lawmakers that the CEO of Cambridge Analytica (CA) met with senior Palantir employees several times to “work on that data,” referring to data used to target Facebook users for political ads. The following day, Palantir provided a statement to the New York Times saying that their employee “in 2013-2014, engaged in an entirely personal capacity with people associated with Cambridge Analytica.” (LINK) This could be true, as Palantir has had no official contract with CA, or this may be a smokescreen to quiet discord and hide something. It is already known that CA was working on some less than ethical projects. Later in 2018, employees of Palantir confronted their CEO, Alex Karp, with complaints about the company’s involvement with the Immigration and Customs Enforcement agency. Though not explicitly stated, it is very likely that ICE was using Gotham or Foundry to search for and then apprehend illegal immigrants working and living in the United States. The concerns were addressed in an internal town-hall meeting and Karp went on to renew the $42 million contract.

Google faced a similar situation in 2018. Google took a step to not seek a renewal of its contract with the US Department of Defense for Project Maven. The project was commissioned by the DoD in 2017 to analyze drone video and other imaging systems using artificial intelligence with the stated goal of identifying simple objects like cars and people. However, such technology could be taken one step further to identify targets, and only a step beyond that is an AI self-authorizing firing weapons. ( When was the last time you saw “The Terminator”?) Google chose to drop the contract at least partly due to over 3000 employees signing a petition to protest the company’s participation. Thiel and Karp decried this decision, and Thiel went on to call Google’s action “treasonous.” Palantir later picked up this contract for the DoD and presumably continued where Google left off.

As far back as 2011, Palantir has faced criticism for its practices. That year, some of its internal emails with partner security firm HBGary were leaked by the hacker collective Anonymous. According to the emails, intelligence companies, including Palantir, were recruited by a law firm to propose ways of sabotaging WikiLeaks on behalf of Bank of America. A month into the talks, US defense and intelligence contractor titan Booz Allen Hamilton was added as a consultant. More leaked data “suggests targeting WikiLeaks’ supporters, including Salon.com journalist Glenn Greenwald.” (LINK) A few days later, Alex Karp said in a statement, “Palantir Technologies do not build software that is designed to allow private sector entities to obtain non-public information, engage in so-called ‘cyber attacks’ or take other offensive measures. I have made clear in no uncertain terms that Palantir Technologies will not be involved in such activities.” This sounds like an oblique admission that the leaked information is at least partially true. (LINK)

Palantir Technologies Inc. began trading Wednesday on the New York Stock Exchange. NYSE

Palantir are always looking for ways to expand their business, and the (first) year of Covid-19 pandemic is not an exception. From the moment of the declared pandemic to October, they have secured $42 million worth contracts with US federal agencies to provide expertise and software to track Covid deaths, testing, vaccine manufacture and distribution, medical supplies and other metrics from dozens of sources. Their new system is named Tiberius, apparently after the character James Tiberius Kirk from the original Star Trek and not the 1st century Roman emperor. The department of Health and Human Services stated that Palantir was a sub-contractor, and would not have direct access to protected health information. Despite this flat assurance, several Democratic senators wrote a letter in July, stating, “Neither HHS nor Palantir has publicly detailed what it plans to do with this PHI (Protected health information), or what privacy safeguards have been put in place, if any. We are concerned that, without any safeguards, data in HHS Protect (Health & Human Services) could be used by other federal agencies in unexpected, unregulated, and potentially harmful ways, such as in the law and immigration enforcement context.” (LINK) On December 7th, Bloomberg reported Palantir won a three-year contract with the Food and Drug Administration worth $44 million to provide software to “integrate and analyze data,” bringing the new Covid arm of the business to $86 million in contracts this year. Palantir’s stock increased by 17% the same day. It seems the pandemic is good for their business.

The Cyber Operations Center on Fort Gordon, Ga., is home to signal and military intelligence non-commissioned officers, who watch for and respond to network attacks from adversaries. (U.S. Army photo)

The last twenty years have seen astronomical growth in internet usage, with social media growing the same, if not more, in the last fifteen years. Unlike television, plain old telephone systems, and newspapers, communicating through and accessing information on the internet provide the server with information about the viewer. In the last four years, we have seen mass-bans of selected individuals, organizations, and even ideas from social media platforms and other internet services like blogs and domain registration. The ostensibly separate companies seem to be coordinating actions. Some collect data on who talks to whom, how often, through what channels, and what is said. Some collect data on how often someone reads which websites, what was liked or upvoted, and what was read. Some collect data on where you are and even what is said in the room through the cell phone you carry with you. Others collect data on your financial history and healthcare. All your interactions with the system are collected by hundreds of ‘cyber-gatekeepers’ as you leave a trail, or digital footprint, across the internet. We must assume this data is stored indefinitely. Add in the prospect of bad actors fabricating data that would be indistinguishable from the real thing, and you can see the implications.

In the hands of a benevolent group, such as the National Center for Missing and Exploited Children or the World Food Bank, Gotham and Foundry can be, and probably are, used to allow organizations to do good and work more efficiently. However, in the hands of the CIA, FBI, FDA, DoD and other alphabet agencies, the morality and benefits to society are less clear. The track record of such agencies speaks for itself. We know personal data is easily available to alphabet agencies through flimsy and infinitely repeatable warrants and the court system the US government created to check and balance this power is little more than a rubber stamp. The power of Palantir’s tools combined with this mass of data is extraordinary. Gandalf would warn us of the dangers of such informational power in the hands of a less than neutral user. However, we cannot “un-invent” this kind of technology. Even if Palantir went out of business, the idea of, and therefore the demand for, big data analysis is a part of our digital society. Palantir’s “chosen sides” statement echoes that of G. W. Bush in 2001: “You’re either with us or against us.”