Beyond the Hype: Mind The Gap

Building the Bridge to your Data with Fabric AI Skills

Data Gravity. The latest term used to depict places where people store data at scale. However, unlike Data Warehouse or Data Lake, Data Gravity invokes images of planet-scale analytical capabilities, and the notion that a valuable repository of data will attract further data sources into its orbit. But, as the laws of data gravity ensure information becomes increasingly co-located, users of this data need increasingly powerful tools to access their information. Also, people drawn to this data source will have increasingly different skill sets and objectives, so flexibility and intuition are essential.

AI Skills in Microsoft Fabric

Microsoft Fabric is an all-encompassing data storage and analytics platform that naturally boasts substantial data gravity, but how can users overcome this force and find the data they need? In the world of Microsoft Fabric, we have AI skills to help us navigate this landscape. AI Skills are powered by Generative AI, coupled with a definitive grounding on your data and its structure to provide an ability to translate natural language questions from your business, into the relevant code needed to generate an accurate response.

Fabric AI Skills are extremely flexible to your data, and can pivot the queries that are written based on the way the data is stored and how it is structured. Users seeking information can ask a question in natural language and the AI Skill will automatically determine the appropriate data source and then write the query in the language that is most suited to that type of data. For example, if the data is derived from a real-time source, the query will be written using KQL. Conversely, if the data sits inside a Lakehouse, an SQL query will be written. Additionally, DAX queries can also be constructed if the data resides in a Semantic Model.

Conveniently, multiple data sources can be attached to an AI Skill, so interconnected concepts like customer or product don’t have to be tackled separately. However, rather than dilute the accuracy of an AI Skill by opening it up to an entire Lakehouse or Semantic Model, developers can select specific tables from the various models to provide perspectives on data that fit the needs of the business.

Additional customisation can be implemented by providing example questions and corresponding queries to the AI Skill, essentially showing the generative AI agent the preferred way to format a query or navigate a set of joins when asked similar questions. Also, instructions can be provided using natural language to guide the bot on certain topics or explain organisational terminology. For example, explaining to the AI Skill that financial data should come from a semantic model, or that log type questions should go to a KQL database.

Usage and Limitations of AI Skills

The AI Skills that are created can be shared in a similar way you might share a document via OneDrive. By generating a link, a user can navigate into Fabric and start asking questions straight away. Bear in mind that a user’s access rights will carry through when using the AI Skill, so ensure grants are configured correctly. Developers have the ability to publish the skill to ensure versions that are still being tested and validated are not inadvertently released, and of course, as with all items in Fabric you can assign endorsements and descriptions to help people find the AI Skills they need.

Fabric AI Skills are continuing to evolve, and so there are certain aspects and limitations to be aware of when creating AI Skills. Mainly, AI Skills will only create “read” queries using SQL, DAX or KQL, it currently will not generate update, create or delete queries and cannot be used across unstructured data in ways that Generative AI typically would. Also, it cannot reason over the data in a way that Power BI Copilot might, by flagging trends or providing explanations for example. Lastly, developers must manually select the available tables / data sources, AI Skills will not seek out relevant data across the entire Fabric eco-system.

Summary

Hopefully, AI Skills will ensure that Fabric has routes for non-technical users to locate and analyse data, in ways that have been pre-optimised and refined by developers who understand the data at a deep level. However they are hardly likely to completely replace user demand for Excel spreadsheets and Power BI dashboards, so we will need to be clear on their best usage and limitations, at least initially. But as Fabric develops, so will AI Skills, and perhaps we will see generative AI providing the bridge into our data ecosystem once and for all.