Thinking Fast, Thinking Slow! Combining Knowledge Graphs ...

1 Thinking fast , Thinking slow ! Combining Knowledge Graphsand Vector of Maryland, BaltimoreCounty1000 Hilltop Cir, Baltimore, Maryland,USA, 21250 Anupam of Maryland, BaltimoreCounty1000 Hilltop Cir, Baltimore, Maryland,USA, 21250 Tim of Maryland, BaltimoreCounty1000 Hilltop Cir, Baltimore, Maryland,USA, 21250 ABSTRACTK nowledge Graphs and vector space models are both robust knowl-edge representation techniques with their individual strengths andweaknesses. Vector space models excel at determining similaritybetween concepts, but they are severely constrained when evaluat-ing complex dependency relations and other logic based operationsthat are a forte of Knowledge Graphs . In this paper, we proposetheV-KG structurethat helps us unify Knowledge Graphs and vec-tor representation of entities, and allows us to develop powerfulinference methods and search capabilities that combine their com-plementary strengths.

2 We analogize this to Thinking fast in vectorspace along with Thinking deeply and slowly by reasoning overthe Knowledge have also created a query processing engine that takes com-plex queries and decomposes them into subqueries optimized to runon the respective Knowledge graph part or the vector part of show that the V-KG structure can process specific queries thatare not efficiently handled by vector spaces or Knowledge also demonstrate and evaluate the V-KG structure and thequery processing engine by developing a system calledCyber-All-Intelfor Knowledge extraction, representation and querying inan end-to-end pipeline grounded in the cybersecurity Representation, Knowledge Graphs , Vector Space Mod-els, Semantic Search, Cybersecurity Informatics1 INTRODUCTIONK nowledge representation is one of the central challenges in thefield of Artificial Intelligence.

3 It involves designing computer rep-resentations that capture information about the world that can beused to solve complex problems [7]. In this paper, we introducetheV-KG structure. It is a hybrid structure which is part knowledgegraph and part embeddings in a vector space. The structure createsa new representation for relations and entities of interest. In theV-KG structure, the Knowledge graph includes explicit informationabout various entities and their relations to each other accordingto an ontology schema. The vector embeddings on the other handinclude implicit information found in context where these entitiesoccur in a corpus [20, 25].While representing Knowledge through different methods thereis an inherent information loss. For example, by representing knowl-edge as just vector embeddings we lose the declarative characterof the information. While Knowledge Graphs are adept at assertingdeclarative information, they miss important contextual informa-tion around the entity or are restricted by the expressibility of thebaseline ontology schema used to represent the Knowledge [7].

4 While discussing the various pros-and-cons of Knowledge graphsand vector space models, it s very important to highlight the factthat both of these Knowledge representation techniques provideapplications built on these technology s advantages. Embeddingsprovide an easy way to search their neighborhood for similar con-cepts and can be used to create powerful deep learning systems forspecific complex tasks. Knowledge Graphs provide access to versa-tile reasoning techniques. Knowledge Graphs also excel at creatingrule-based systems where domain expertise can be overcome limitations of both so as to take advantage of theircomplimentary strengths, we propose the V-KG structure that ispart Knowledge graph and part vector embeddings. Together, theycan be used to develop powerful inference methods and a bettersemantic this paper, we present a method to create the V-KG struc-ture from a given textual corpus.

5 We also discuss special types ofqueries that can take advantage of this structure, specifically thequeries that are decomposed automatically into sub-queries thatrun efficiently on the vector space part and the Knowledge graphpart. We also discuss some use-cases and applications that can takeadvantage of the V-KG demonstrate and evaluate the V-KG structure and the queryprocessing engine we have developed a system calledCyber-All-Intel. Our system strives to bring in intelligence" about the stateof the cyber world, such as an analyst might obtain by looking, forinstance, at security blogs, NVD/CVE updates, and parts of the darkweb. It represents this cybersecurity informatics Knowledge in theV-KG structure. We then use the system to answer complex cyberse-curity informatics queries and process tasks like, Issue an alert if avulnerability similar to denial of service is found in MySQL , which in-volves various operations on the V-KG structure namely, searchingfor similar vulnerabilities, querying to find known vulnerabilitiesin a product, and reasoning to come up with an main contributions of this paper are: TheV-KG structurethat is part vector embeddings and partknowledge [ ] 10 Aug 2017 A query processing engine that decomposes an input querytosearch,list, andin f eron the V-KG structure.

6 A cybersecurity informatics system built on top of the V-KGstructure rest of the paper is as follows In Section 2, we discussthe related work. Section 3 gives various details about the V-KGstructure. We present our evaluation in Section 4, we conclude inSection RELATED WORKIn this section we present some related work in the field of knowl-edge Graphs , vector space models, and Knowledge representationin Vector Space Models & Knowledge GraphsWord embeddings are used to represent words in a continuousvector space. Two popular methods to generate these embeddingsbased on Relational Concurrence Assumption are word2vec [20,21] and GloVe [25]. The main idea behind generating embeddingsfor words is to say that vectors close together are semanticallyrelated. Word embeddings have been used in various applicationslike machine translation [33], improving local and global context[13], Knowledge Graphs assert facts in the form of (Subject,Predicate,Object) triples, whereSubjectandObjectare modeledas graph nodes and the edge between them (Predicate) model therelation between the two.

7 DBpedia [2], YAGO (Yet Another GreatOntology) [32], YAGO2 [12], Google Knowledge graph [28], some of the examples of popular Knowledge important task on both vector space models and knowl-edge Graphs is searching for similar entities, given an input vector spaces, embeddings close together are semantically re-lated and various neighborhood search algorithms [10,17] havebeen suggested. On the other hand semantic similarity on knowl-edge Graphs using ontology matching, ontology alignment, schemamatching, instance matching, similarity search, etc. remains a chal-lenge [8,27,41]. In this paper we propose the V-KG structure, inwhich we link the Knowledge graph nodes to their embeddings ina vector space (see Section 3).Yang et al. [40] argued that a fast top-k search in knowledgegraphs is challenging as both graph traversal and similarity searchare expensive. The problem will get compounded as knowledgegraphs increase in size.

8 Their work proposes STAR, a top-k knowl-edge graph search framework to find top matches to a given et al. [6] have suggested using Random Indexing (RI)to generate a semantic index to an RDF1graph. These factors com-bined have led to an increased interest in semantic search, so as toaccess RDF data using Information Retrieval methods. We arguethat vector embeddings can be used to search, as well as indexentities in a Knowledge graph . We have built a query engine ontop of the V-KG structure that removes the need to search on theknowledge graph and uses entity vector embeddings instead (seeSection 3 and 4). However, queries that involve listing declarativeknowledge and reasoning are done on the Knowledge graph partof the V-KG Knowledge Graphs have also been created, systemslike HOLE (holographic embeddings) [24] and TransE [39] learncompositional vector space representations of entire knowledgegraphs by translating them to different hyperplanes.

9 Our workis different from these models as we keep the Knowledge graphpart of the V-KG structure as a traditional Knowledge graph so asto fully utilize mature reasoning capabilities and incorporate thedynamic nature of the underlining corpus for our cybersecurityuse-case. Vectorizing the entire Knowledge graph part for a systemlike Cyber-All-Intel will have significant computational overheadbecause of the ever-changing nature of vulnerability relations andvelocity of new input threat another work thread different from ours, vector models havealso been used for Knowledge graph completion. Various authorshave come up with models and intelligent systems to predict if cer-tain nodes in the Knowledge Graphs should have a relation betweenthem. The research task here is to complete a Knowledge graph byfinding out missing facts and using them to answer path queries[11, 18, 23, 30].

10 Knowledge Representation forCybersecurityKnowledge Graphs have been used in cybersecurity to combine dataand information from multiple sources. Undercofer et al. createdan ontology by Combining various taxonomies for intrusion detec-tion [38]. Kandefer et al. [16] created a data repository of systemvulnerabilities and with the help of a systems analyst, trained asystem to identify and prevent system intrusion. Takahashi et al.[35,36] built an ontology for cybersecurity information based onactual cybersecurity operations focused on cloud computing-basedservices. Rutkowski et al. [26] created a cybersecurity informationexchange framework, known as CYBEX. The framework describeshow cybersecurity information is exchanged between cybersecurityentities on a global scale and how implementation of this frameworkwill eventually minimize the disparate availability of et al.