https://tyk.io/blog/robust-ai-governance-is-critical-to-enterprise-adoption-and-implementation/ * Platform + For Teams o API platform management o API product management + FOR DEVELOPERS o API lifecycle management o Design, test and publish APIs o Secure and transform traffic o No-code API integration o Gather and export analytics + Explore More o Demo library o Tyk vs the competition o Open source API gateway * Deployment + Tyk CloudSecure, publish and maintain APIs at any scale, anywhere in the world. + Self-ManagedTake charge of your REST and GraphQL APIs on your terms. + Multi-cloud and hybridPowerful, flexible API management for multi-cloud and hybrid environments + Multi-regionDeploy your APIs through gateways deployed in every location you need + KubernetesGitOps practices to API Management processes + GitOpsEasy API management with GitOps focused solution * Pricing * Community + Latest Insights o Tyk Community o Customer case studies o Events o Whitepapers o Reviews o Podcast o Blog + NEWSLETTERS o API platform engineering o API product management * Company + About Tyk + Careers + Working at Tyk + 24/7 Support + Partners + Contact us * Contact us * Get Started * Docs * Blog * Cloud login Tyk API ManagementTyk API Management Tyk API ManagementTyk API Management Robust AI governance is critical to enterprise adoption and implementation Publish on 21 Nov, 2023 - by Martin Buhr Last updated: 21 Nov, 2023 AI Announcement API Strategy Platform Engineering Product Management [gif][blog-header-montag] AI seems to be everywhere in the tech space at the moment. You can't avoid it, it's at the peak of the hype-cycle, on the cusp of sliding into the trough of disillusionment. But will it? At the pace AI vendors are innovating and improving, the hype-train might outrun the hype-cycle to actually live up to its own expectations. Who really knows? AI is everywhere, and it is hyped What we do know is that AI, specifically generative AI (that's the one that can ace the bar exam, generate works of art in the style of any artist, and mimic our voices and faces with ease) is being eyed by every industry as a path to increased productivity. This is a good thing. I'm pretty sure everyone would like to have an easy, guilt-free helping hand to get them through their duller tasks, writing repetitive code, structuring a quick social media post or outlining a blog post. Generative AI is incredibly powerful when it comes to assisting folks who are already good at their jobs at being even more effective. The challenges facing organisations that want to use AI What is clear is that to really embrace the productivity gains and potential cost-savings from AI, organisations need to really get under the hood of what it means to use and implement an AI strategy. In general, an individual using ChatGPT, or DALL-E, or Stable Diffusion, is not so concerned about the information they are sending out into the jaws of the language model. And, more importantly, neither are they concerned about the black-hole of data consumption that is its training and benchmarking data lake owned by the vendor. The very data that is critical to consistently improving the quality of the underlying technology. For an organisation to adopt AI, this poses a real issue. Not just from a confidentiality and trade secrets perspective, but from a boring old data governance, privacy, and accreditation perspective. Are you an ISO9001 organisation? AI-as-a-Service is a problem. Are you a PII or financial processor (e.g. HIPAA or PCI)? AI-as-a-Service is a problem. Are you ISO 27001 certified? Nope, not for you! SOC2? Ah-ah-ah, you didn't say the magic words! Are you in the EU and allowing folks to sign up using their real name? Oof! GDPR might get you in trouble when it comes to AI. What I'm trying to say is that adoption of AI into your organisation comes with a plethora of pitfalls when it comes to overall data governance. Key challenges when adopting AI tools A common counter-argument is that ultimately, using AI as a Service is very similar to using services like Google Drive, Amazon S3 or Azure Cloud to store documents. However it isn't, really. You have a responsibility to your customers to safeguard their data, and in turn a repository of data that is not processed or read by the vendor is not the same as a repository of data that is eventually literally baked into the vector-matrices of a large-language model. The key challenges that face an organisation when wishing to adopt AI are: 1. What data am I going to make available to the AI? This isn't just initial training data, or contextual data ("embeddings" in the lingua-franca of the biz), but even just the input question itself: imagine sending a customer support ticket directly to an AI - that's your customer's name, email address, plus whatever is in the support email body itself that could be PII or sensitive. 2. Who will be using these AIs in my organisation? What level of data access do they have? 3. How will my staff be using these AIs? Are they using them for the good of the company or are they abusing (sometimes expensive) access for personal gain? 4. How can AI Hallucinations be controlled? AI algorithms are opaque and can result in unexpected outputs. How can my organisation best protect against these creating negative outcomes? The answer: Robust Governance. Introducing Montag, Tyk's AI governance tool Now, I am pretty stoked about AI, so much so that I wrote a memo to the wider company telling our folks to go off and embrace AI tools, to use their good judgement and only focus on things that are public facing (I then promptly got told off by both Legal and IT). Regardless, the genie was out of the bottle, and we needed to get serious. So we built Montag. Montag.ai is our internal AI governance tooling used by our Platform Team, it enables us to to: * Centralise access credentials to AI vendors, be that commercial offerings or in-house ones that we run ourselves on our own hardware * Log and measure how AI is being used internally * Build and release Chatbots for our internal Slack channels * Ingest data into our knowledge corpus, with siloed access settings * Build AI functions (Smart Functions) as a Service for common, single-task automations (e.g. summarise this article) or modulated pass-through to upstream AI's with query inspection to provide context-aware access control. * Script complex multi-modal assistants that can selectively determine which AI vendor, model, and data set to use depending on the nature of the user input * Full-blown RBAC to ensure we can democratise access to LLMs without impinging on our security obligations It does a whole bunch more than that, but ultimately it solved our key issues for us: 1. What data am I going to make available to the AI? Our internal assistant (Wadsworth), will identify internal links and data submitted by the end user and direct these solely to self-hosted models. While other sensitive data (not PII), can be sent to vendor AIs that we have an Enterprise agreement with. 2. Who will be using these AIs in my organisation? With fine-grained access controls, we can ensure that only specific users have direct access to data repositories (and can write and read from them), and by providing pass-through endpoints for raw AI access (e.g. a developer wants to PoC on GPT-4), we can provide specific, monitored internal API endpoints and internal API credentials to provide safe access. 3. How will my staff be using this AI? Access to smart functions, AI endpoints, and bots can be strictly controlled through a combination of RBAC, and internally generated API tokens. 4. How can AI hallucinations be controlled? By using an AI bot pipeline to verify generated content through a sequence of bots designed to specifically validate content. We have our primary Q& A bots use another bot to check their answer and rewrite it. For API-level access, our platform team will use our internal Tyk APIM stack to provide tokens, rate limits and endpoint-specific access to the Montag Smart APIs. Which brings me to my next point... How we see the intersection of APIM and AI, in the most realistic case [gif][AI-management-35-934x228] AIs are APIs. While chat is seen as the universal interface, what makes AI so powerful is the programmatic access - whether it's a consumer using AIs to enable a new capability, or the AI's themselves calling APIs to enhance their own responses (viz. ChatGPT Function calling), and this functionality is fundamentally powered by APIs. Specifically: internal APIs, APIs that give access to your customer data. Your sales data, your warehouse inventory - whatever it takes to make the AI as useful, as productive, and as accurate as possible. While many organisations in the API management space are focussing on shoe-horning AI into their products, we very much see AI as an enabler for business value, and APIM as an enabler for that. A significant factor in the recent rapid adoption of AI tools has been the method through which they are packaged and consumed, with APIs playing a central role. API management enables AI-as-a-Service The intersection of APIM and AI is that platform teams should be using APIM to provide AI-as-a-Service to their internal customers, and should be using APIM to mediate and monitor how their products - enhanced with AI - provide value to their customers. So, instead of announcing some shoe-horned AI feature for our APIM stack, we're instead going to release our Montag.ai stack as a product to empower platform teams and product teams adopt AI quickly, easily, and most importantly, ensure robust AI governance for their organisations. [gif][blog-banner-montag] --------------------------------------------------------------------- Share with your network logo twitterlogo twitter logo linkedinlogo linkedin logo facebooklogo facebook logo redditlogo reddit logo Hackernewslogo Hackernews Related posts * Do AI chatbot plugins spell the end of UX as we know it? * What do ChatGPT Plugins mean for API management? Recent posts * The API product management newsletter #015 - 21 November 2023 * Robust AI governance is critical to enterprise adoption and implementation * Unleashing the power of Tyk for start-up platform engineering teams * Simplifying proof of concept: Running Enterprise Portal and Tyk Pro in Docker * 28 ways brands are building world-leading API experiences Want to get in touch? Feel like you need a bit more help with your choice? Contact us and we'll happily help you further. Speak to our team for more API expertise. Get in touch * Extra + Cloud status + Raise a bug * Legal + Terms & Conditions + Privacy Policy + GDPR + Responsible Disclosure Subscribe to our mailer Github LinkedIn Twitter Facebooks Youtube Podcast * amazonpartner-logoamazonpartner-logo * UKAS-ISO9001-Mark-cl-27_Mono-aiUKAS-ISO9001-Mark-cl-27_Mono-ai * UKAS-ISO27001-Mark-cl-27_Mono-aiUKAS-ISO27001-Mark-cl-27_Mono-ai * pcipci * aicpa-soc-logoaicpa-soc-logo * Flexa-seal-white-footerFlexa-seal-white-footer * infocomm-media-development-authority-imda-logo-vector -white infocomm-media-development-authority-imda-logo-vector -white (c) Tyk Technologies, 2023 Click to Copy We use cookies to make this website better, improve your experience and offer you personalised content. If you wish to opt-out from the use of cookies please see our Cookies Policy or Reject all no essential ones. SettingsReject Accept all Privacy & Cookies Policy Close Privacy Overview This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are as essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience. Necessary [*] Necessary Always Enabled Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information. Analytics [ ] analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc. Functional [ ] functional Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features. Performance [ ] performance Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors. Non-necessary [ ] non-necessary Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website. Save & Accept Powered by CookieYes LogoCookieYes Logo