https://github.com/rectanglehq/Shapeshift Skip to content Navigation Menu Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + GitHub Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions By size + Enterprise + Teams + Startups By industry + Healthcare + Financial services + Manufacturing By use case + CI/CD & Automation + DevOps + DevSecOps * Resources Topics + AI + DevOps + Innersource + Open Source + Security + Software Development Explore + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Enterprise + Enterprise platform AI-powered developer platform Available add-ons + Advanced Security Enterprise-grade security features + GitHub Copilot Enterprise-grade AI features + Premium Support Enterprise-grade 24/7 support * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} rectanglehq / Shapeshift Public * Notifications You must be signed in to change notification settings * Fork 4 * Star 257 Transform JSON objects using vector embeddings 257 stars 4 forks Branches Tags Activity Star Notifications You must be signed in to change notification settings * Code * Issues 0 * Pull requests 0 * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Security * Insights rectanglehq/Shapeshift This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main BranchesTags Go to file Code Folders and files Name Name Last commit Last commit message date Latest commit History 7 Commits .gitattributes .gitattributes example.ts example.ts graphic.png graphic.png index.ts index.ts package.json package.json readme.md readme.md rectangle-logo.png rectangle-logo.png View all files Repository files navigation * README [rectangle-logo] Shapeshift Shapeshift is a TypeScript library that maps arbitrarily structured JSON objects using vector embeddings. It uses semantic similarity to match keys between objects, allowing for flexible and intelligent object transformation, including support for nested structures. [graphic] Features * Map objects with different structures based on semantic similarity of keys * Support for nested objects * Multiple embedding providers: Cohere, OpenAI, and Voyage * Customizable embedding model and similarity threshold * TypeScript support for type safety Installation npm install @rectanglehq/shapeshift Ensure you have an API key from your chosen embedding provider (Cohere, OpenAI, or Voyage). Usage Here's an example demonstrating Shapeshift's capability to handle nested objects: import { Shapeshift } from "@rectanglehq/shapeshift"; const sourceObj = { personalInfo: { name: "John Doe", age: 30, }, occupation: "Software Engineer", fullAddress: "123 Main St, Anytown", address: { street: "123 Main St", city: "Anytown", }, }; // The values of the target object are irrelevant const targetObj = { fullName: "", yearsOld: 0, profession: "", location: { streetAddress: "", cityName: "", }, }; const shapeshifter = new Shapeshift( { embeddingClient: "cohere", apiKey: process.env.COHERE_API_KEY || "" }, { embeddingModel: "embed-english-v3.0", similarityThreshold: 0.5 } ); const shiftedObj = await shapeshifter.shapeshift(sourceObj, targetObj); console.log("Shifted object:", shiftedObj); API Shapeshift Class Constructor new Shapeshift( { embeddingClient: EmbeddingClient, apiKey: string }, options?: ShapeshiftOptions ) * embeddingClient: The embedding provider to use ('cohere', 'openai', or 'voyage'). * apiKey: Your API key for the chosen embedding provider. * options (optional): + embeddingModel: The embedding model to use (default varies by provider). + similarityThreshold: The minimum similarity score to consider a match (default: 0.5). Methods shapeshift(sourceObj: T, targetObj: U): Promise Maps the sourceObj to the structure of targetObj based on semantic similarity of keys, including nested structures. * sourceObj: The source object to map from (can include nested objects). * targetObj: The target object structure to map to (can include nested objects). * Returns: A promise that resolves to the shifted object with the structure of targetObj. Supported Embedding Providers and Models Cohere * Default model: 'embed-english-v3.0' OpenAI * Default model: 'text-embedding-ada-002' Voyage * Default model: 'voyage-large-2' How It Works 1. The library flattens both the source and target objects, preserving the nested structure information in the keys. 2. It calculates embeddings for the flattened keys of both objects using the specified embedding provider and model. 3. For each key in the flattened source object, it finds the most semantically similar key in the flattened target object using cosine similarity of their embeddings. 4. If the similarity score is above the specified threshold, the value from the source object is mapped to the corresponding key in the target object. 5. Finally, the library unflattens the result, restoring the nested structure of the target object. Handling of Nested Objects Shapeshift can handle nested objects in both the source and target structures. It does this by: 1. Flattening nested objects into a single-level object with dot-notation keys. 2. Performing the semantic matching on these flattened keys. 3. Reconstructing the nested structure in the final output. This allows for flexible mapping between different object structures, even when the nesting doesn't exactly match. Limitations * The quality of mapping depends on the semantic similarity of key names. Very dissimilar names may not map correctly. * While the library can handle nested objects, extremely deep nesting may impact performance. Contributing Contributions are welcome! Please feel free to submit a Pull Request. License This project is licensed under the MIT License. About Transform JSON objects using vector embeddings Resources Readme Activity Custom properties Stars 257 stars Watchers 2 watching Forks 4 forks Report repository Releases No releases published Packages 0 Languages * TypeScript 100.0% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.