Every company is becoming a data company. In Data-Powered Businesses, we dive into the ways that companies of all kinds are digitally transforming to make smarter data-driven decisions, monetize their data, and create companies that will thrive in our current era of Big Data.
At Sisense, we help organizations thrive in a data-driven world by making their data easily accessible to as many people as possible. We empower companies of all kinds to get the most out of their digital transformation and derive maximum benefit from the data they generate and gather.
That’s not always easy. However, Sisense’s natural language query (NLQ) interface, combined with content recommendations from knowledge graphs, can help users of all technical skill levels get more from their data.
We caught up with Dr. Yael Lev, to learn more about knowledge graphs and the role they play in the democratization of data. Yael is a member of the AI Research team at Sisense and she is the tech lead for Sisense’s knowledge graph project.
Adam Murray: Tell us what you do.
Yael Lev: I work closely with my colleagues to constantly improve the Sisense platform’s data and analytics capabilities. We aim to enable users to build and consume AI applications for augmented analytics, automatic data preparation, and conversational data exploration.
Specifically, I work on knowledge graphs. They recommend and expand queries, thereby improving the users experience of Sisense NLQ, which allows users to ask questions of their data in straightforward language and interact with it themselves. Sisense’s recommendation engine delivers accurate responses to queries and suggests new ideas that enhance data analysis.
AM: What kind of recommendations can users get?
YL: Generally, there are two kinds of recommendations: Recommendations for new questions the other is autocomplete options for queries. Question recommendations point users to popular queries that arise from the knowledge graph. Autocomplete options happen while users type into the NLQ search bar. For example, writing “total sales” will trigger recommendations to further refine or filter your search, such as by country, or by date. The more you type, the more recommendations we can make to create a more specific question.
AM: How is data collected to make these recommendations?
YL: We’re interested in metadata. We want to know what columns and tables customers are querying, and the structure of that query. Behind the scenes, a widget generates a query in ElastiCube, our high-performance analytics database, and this goes into a pipeline that’s accumulated in the graph.
Queries can be broken out into different items: For example, total sales will be one query item, country a second, gender a third, and so on. Each query item can be broken into columns. Each column is part of a data source, and a Sisense environment, so we take apart all these building blocks of a single query, and you get different types of graph nodes and relationships. A graph is essentially a map of the query, showing its constituent parts.
AM: What enables them to make these connections?
YL: Well, we classify information into node types. A few examples of node types are User ID, User Group, Query, and Widget. By identifying these nodes and connecting them with a relationship on the graph, we can understand how users are interacting with our platform. Based on that, we give more personal and customized recommendations to each user.
AM: Can you explain a little more how knowledge graphs handle queries and make recommendations?
YL: Let’s say you search for “total sales by gender.” That’s two query items: total sales and grouping by gender. This straightforward recommendation is a fully contained recommendation that has strict parameters. Find all the queries that contain these two items and see what else was asked with it, and we will return them. In the knowledge graph, we have weights of usage depending on how frequently you ask a question, so we can make recommendations based on popularity.
Partially contained recommendations are similar but don’t have to be a full match, so we’re actually creating new queries. By mapping relationships across dashboards, we’re expanding the questions that can be asked and offering recommendations that you might not have considered.
AM: So as the platform is used more, it learns. Is that right?
YL: Yes. The knowledge graph actively learns, registering when the user chooses to accept a recommendation, and ultimately delivering more personalized suggestions. Recommendations from the knowledge graph are scored so it learns which are most relevant. Also, if a user likes recommendations further down the list, then these recommendations should move up for subsequent queries. So, we adjust our score to know how recommendations can improve for that user in future. This technique is called the “reward function.”
Another great thing about knowledge graphs is that they serve as a kind of organizational memory. In every organization, people change their roles or move on. New people interact with the data. They get recommendations based on what predecessors have asked, and others in their user group, so the knowledge of any previous data searches isn’t lost and can inform recommendations.
Sisense: You’ve previously written that every company should have a knowledge graph. Why?
YL: Knowledge graphs are the basis for good BI. They’re where you query databases, capture relevant searches, and they easily aggregate all the usage in a way that’s very easy to analyze. They help make data more accessible and provide insights for everyone.
AM: So, just to clarify, would you say that the knowledge graphs help improve queries and get better insights?
YL: Yes. To get the most relevant answers you’ve got to ask the right questions, and if something helps you ask better questions, then you’re more likely to get what you need.
Looking at the way we build our graph, it accumulates usage patterns. If we wanted to query a database holding all this data instead of a graph, we would have to perform aggregation, but here we don’t because the way we build the data model, the aggregation is on the inside. If you wanted to find out how many times a certain question was asked, you wouldn’t need to count all the rows for this question. You simply find that question and the number will be there. It’s a much more accessible way of generating the most relevant queries.
AM: You’ve also previously written about how knowledge graphs help break down data silos, find information fast, make better decisions, and uncover hidden insights. Can you expand on that?
YL Well, amalgamating naturally different silos —data from different sources or contexts — can be a heavy calculation, because you would have to scan the whole database, you could have one table that talks about the data structure, and another that talks about the user activity, and those two don’t necessarily connect. In a knowledge graph, they’re connected by a single relationship, and you can join your worlds (providing there’s one component that connects them). Then it’s just a matter of one or two hops between nodes. It’s a much lighter computation.
Regarding better decision-making: If it’s easier to ask more complicated questions, then you get answers faster and you can make better, more timely decisions. Plus, when you connect different data silos, you can get insights that maybe you wouldn’t have previously. You uncover hidden insights and gain a more comprehensive understanding of relationships between data.
AM: What do you think are the obstacles for organizations struggling to adopt data and analytics and how can we help them? What role can knowledge graphs play?
YL: I think connecting data siloes is a challenge because you can have data modeled in a relational database, but you still have to connect it all. Knowledge graphs make this task easier, faster and much less of a strain on resources. In the Sisense platform, the knowledge graph sits in the back end as an enabler of queries and recommendations, providing the most efficient way to ask questions of data.
So, let’s say a new customer has just come on board with Sisense. They’ve set up some dashboards and want to ask a question. They simply type in their question and (because the knowledge graph behind the scenes enables the NLQ capability), they receive potential answers and recommendations. That’s the knowledge graph in action.
AM: What are the next developments in knowledge graphs that you’re making at Sisense and how will they benefit analytics and digital transformation further?
YL: What we want to do in the future, is to understand better what questions users are asking so that we can hone recommendations. The future of the Sisense knowledge graph capabilities lies in data storytelling: providing our users with the right insights at the right time, packaged into the right visualization for them, and turning this data into a coherent story of their business.
Yael Lev was a featured speaker in our webinar, “Humanizing BI Through Natural Language Processing and Knowledge Graphs,” alongside data scientist Ayelet Arditi and Sisense product marketing manager Julie Zuckerman. Together, they discussed how to make data exploration more intuitive and approachable with natural language, how to build a natural language query pipeline, and more about knowledge graphs. You can click here to view the entire webinar on demand.
Adam Murray began his career in corporate communications and PR in London and New York before moving to Tel Aviv. He’s spent the last ten years working with tech companies like Amdocs, Gilat Satellite Systems, and Allot Communications. He holds a Ph.D. in English Literature. When he’s not spending time with his wife and son, he’s preoccupied with his beloved football team, Tottenham Hotspur.