Expert System has just joined AFCEA –Armed Forces Communications and Electronic Associations, Rome - an international association that aims to promote “active communication between the government, the military, the media, and other types of industry associated with the world of (tele)informatics.”
Semantic Technology and Mathematical Algorithms: from the Invisible to the Visible.
Expert System – Semeion’s Crime Fighting and Homeland Security Solutions
Andrea Melegari, VP Security & Intelligence Expert System
Massimo Buscema, General Manager Semeion
Expert System has developed Cogito, the world’s only technology capable of deep semantic analysis.
Cogito software is able to:
- extract and analyze data contained in thousands of documents, emails, web pages …
- automatically understand the meaning of every word, in every text, written in everyday language (natural language).
Unlike traditional technology, which is run by keywords and statistics and can only attempt to guess the meaning of a text, Cogito is able to read and interpret potentially pertinent knowledge and automatically identify the conceptual relations between various data. It is able to identify the most relevant concepts, the entities, the events and the specific information useful for analysis as well as the main data elements in whichever form they may have been expressed. It normalizes and orders content, generates metadata maps to improve the use of the available information and supports analysts in Intelligence activities.
The algorithms developed by the Semeion Center are able to formulate unique data analyses by applying key intelligent systems able to seek out hidden rules within structured data.
The analysis of hidden information is also known as Intelligent Data Mining (IDM).
IDM is based on complex mathematical algorithms which were developed and patented by Semeion. These algorithms act as artificial adaptable agents which interact with the data in order to discover the existing hidden relationships.
Case study
A Gartner case study estimated that 80% of the information available is presently in a disorganized form. Articles, documents, emails, minutes, web pages are just a few of the different forms of written communication.
Most computer applications however, use databases (organized data). Semeion’s algorithms also require databases.
The extraction, transformation and loading of information (entities and relations between entities) from a text into a database (a process also known as ETL) is a crucial aspect of the entire operation.
Let’s use this news report as an example:
March 2, 2009 (PAKISTAN): A suicide bomber killed six people at a religious school for girls in Balochistan Province. Pakistani press reports stated that the attacker wanted to assassinate a senior leader of Jamiat, who was scheduled to speak at the school The Jamiat leader was not harmed in the attack.
(Taken from the April issue of the Combating Terrorism Center bulletin http://www.ctc.usma.edu/sentinel/).
Many entities and different actions can be found in just these two sentences: criminal organizations, people, provinces, actions, buildings, countries, dates, etc…
A few entities and actions extracted by ETL
The entities are also connected by complex relationships which are characterized by different attributes.
< class="link-lightbox"a href="../img/tecnologie-homeland-02-big.gif" rel="lightbox" title="relationships of “kill” were identified and contextualized by Cogito">
Relationships of “kill” were identified and contextualized by Cogito
Thanks to Cogito’s semantic technology, the ETL process automatically transforms output into XML data.
The XML data can then be easily visualized in graphic form or it can be transposed into a database.
Cogito transforms this same news story
March 2, 2009 (PAKISTAN): A suicide bomber killed six people at a religious school for girls in Balochistan Province. Pakistani press reports stated that the attacker wanted to assassinate a senior leader of Jamiat, who was scheduled to speak at the school. The Jamiat leader was not harmed in the attack.
into a database record.
Cogito’s ETL processes content very rapidly.
and transforms it into a database like this:
Although the sample archive used for this case history was purposely refined to include few records related to terrorism, it is still a valid example which illustrates the various processes and the obtainable results.
In this phase, it is important not to focus solely on the results obtained from this sample, but instead, on the versatility of this semantic-mathematic elaboration and its subsequent application to other areas, such as: the analysis of social networks, the origin and evolution of news, communication between groups, political and economic repercussions from the publication of information, etc...
Once a database has been created, Semeion’s algorithms can then be applied. This cognitive map organizes the concepts which were identified by Cogito.
The cognitive map generated by Semeion’s algorithm
The map illustrates that:
- suicide attacks are: linked to bomb use, their targets are mainly military-related and in Afghanistan and their objective is to kill, especially in Iraq. Also, they do not concern the factions cited in the news story;
- the attacks in Pakistan: are carried out by missiles, at night and involve Talibans, have a political objective when carried out during the day, cause civilian casualties and are often organized by al-Qaida, and in these cases, they are analogous to those which occur in Yemen;
- if these attacks have religious motives, then they are similar those carried out in Somalia, they use professional killers from the two factions cited and cause injuries.
These details aren’t visible within the various information analyzed (and inserted in the XLM stream). They are hidden within the news story as possible and probable inferences. The cognitive map - Auto Contractive Map - used a complex mathematical algorithm to apply abstract inferences to concepts, therefore revealing a world of connections between highly probable concepts.
Semeion’s other algorithms interrogate the database which Cogito created in order to discover the relations which no SQL system is able to find. In fact, in this news story, it is clear that the terroristic attack had religious motives, but did not have military-related motives.
It is possible, however, to find out the characteristics of an attack involving both concepts: Semeion’s Activation and Competition System (ACS) is able to manipulate the small concept base and propose a prototype of a possible attack with both military and religious motives (highlighted in red):
- it could occur in Iraq or in Afghanistan (the ‘white squares’ show that both countries have a high likelihood), maybe in Somalia, but not in Pakistan;
- the use of “missiles” is considered, but then rejected as “bombs” and “suicide attacks” seem more likely;
- there is a connection with “political” objectives, which implies deaths, a daytime occurrence and no civilian victims.
We are dealing with inferences, therefore with probable, but not certain, connections. Life also consists in the ability to formulate plausible hypotheses and forecasts.
The short news story that we used as an example also contains the location of some of the attacks. Cogito gives us the latitude and longitude of each attack, but these are not enough to formulate a hypothesis on what we really want to know: if the attacks are indeed carried out by one single organization, where might the probable headquarters for such an organization be located?
Semion’s dHarmonic algorithm is able to guess the “hidden point” from where the attacks possibly originated, using only the attack points which were captured by Cogito from the disorganized news story.
The dHarmonic algorithm is able to make hypotheses of this kind for one reason: acts of man, when organized and/or repeated over time, always contain a hidden order.
This kind of spontaneous organization also occurs in nature (the map above and below illustrate the results of this process).
The probable location of the headquarters is identified on a map of Afghanistan