https://github.com/davidlatwe/montydb Skip to content Sign up * Why GitHub? Features - + Mobile - + Actions - + Codespaces - + Packages - + Security - + Code review - + Project management - + Integrations - + GitHub Sponsors - + Customer stories- * Team * Enterprise * Explore + Explore GitHub - Learn and contribute + Topics - + Collections - + Trending - + Learning Lab - + Open source guides - Connect with others + The ReadME Project - + Events - + Community forum - + GitHub Education - + GitHub Stars program - * Marketplace * Pricing Plans - + Compare plans - + Contact Sales - + Education - [ ] [search-key] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} davidlatwe / montydb * Notifications * Star 163 * Fork 8 Monty, Mongo tinified. MongoDB implemented in Python ! BSD-3-Clause License 163 stars 8 forks Star Notifications * Code * Issues 4 * Pull requests 1 * Discussions * Actions * Projects 4 * Wiki * Security * Insights More * Code * Issues * Pull requests * Discussions * Actions * Projects * Wiki * Security * Insights master Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags 3 branches 9 tags Code Clone HTTPS GitHub CLI [https://github.com/d] Use Git or checkout with SVN using the web URL. [gh repo clone davidl] Work fast with our official CLI. Learn more. * Open with GitHub Desktop * Download ZIP Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Go back Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Go back Launching Xcode If nothing happens, download Xcode and try again. Go back Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @davidlatwe davidlatwe Merge pull request #34 from jqueguiner/patch-2 ... 4b97d76 May 29, 2021 Merge pull request #34 from jqueguiner/patch-2 4b97d76 Git stats * 595 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github/workflows Create python-publish.yml May 29, 2021 artwork New ident Jul 28, 2019 montydb version bump May 29, 2021 tests ensure utils testing space May 23, 2021 .coveragerc Improve weighted test cov Jun 1, 2018 .gitignore Update .gitignore Apr 27, 2018 LICENSE Update LICENSE May 22, 2018 README.md Update README.md May 29, 2021 pyproject.toml use setup.cfg Apr 28, 2021 setup.cfg refactor versioning May 29, 2021 setup.py use setup.cfg Apr 28, 2021 View code What ? Install Storage Configuration In-Memory Flat-File SQLite LMDB (Lightning Memory-Mapped Database) URI Utilities montyimport montyexport montyrestore montydump MongoQueryRecorder MontyList Why I did this ? README.md drawing Python package Version Monty, Mongo tinified. MongoDB implemented in Python ! Was inspired by TinyDB and it's extension TinyMongo What ? A pure Python implemented database that looks and works like MongoDB. >>> from montydb import MontyClient >>> col = MontyClient(":memory:").db.test >>> col.insert_many([{"stock": "A", "qty": 6}, {"stock": "A", "qty": 2}]) >>> cur = col.find({"stock": "A", "qty": {"$gt": 4}}) >>> next(cur) {'_id': ObjectId('5ad34e537e8dd45d9c61a456'), 'stock': 'A', 'qty': 6} Most of the CRUD operator has been implemented, you may visit this issue to see the full list. And this project is testing against to: * MongoDB 3.6, 4.0, 4.2 (4.4 on the way) * Python 2.7, 3.6, 3.7, 3.8, 3.9 Install pip install montydb * optional, to use bson in operation (pymongo will be installed) pip install montydb[bson] * optional, to use lightning memory-mapped db as storage engine pip install montydb[lmdb] Storage Available storage engines: * in-memory * flat-file * sqlite * lmdb (lightning memory-mapped db) Depend on which one you use, may have to config the storage engine before start. [?][?] The configuration process only required on repository creation or modification. And, one repository (the parent level of databases) can only assign one storage engine. To configurate a storage, take flat-file storage as example: from montydb import set_storage, MontyClient set_storage( # general settings # repository="/db/repo", # dir path for database to live on disk, default is {cwd} storage="flatfile", # storage name, default "flatfile" mongo_version="4.0", # try matching behavior with this mongodb version use_bson=False, # default None, and will try importing pymongo if None # any other kwargs are storage engine settings # cache_modified=10, # the only setting that flat-file have ) # ready to go Once that done, there should be a file named monty.storage.cfg saved in your db repository path, it would be /db/repo for above examples. Configuration Now let's moving on to each storage engine's config settings. In-Memory memory storage does not need nor have any configuration, nothing saved to disk. from montydb import MontyClient client = MontyClient(":memory:") # ready to go Flat-File flatfile is the default on-disk storage engine. from montydb import set_storage, MontyClient set_storage("/db/repo", cache_modified=5) # optional step client = MontyClient("/db/repo") # use curent working dir if no path given # ready to go FlatFile config: [flatfile] cache_modified: 0 # how many document CRUD cached before flush to disk. SQLite sqlite is NOT the default on-disk storage, need configuration first before getting client. Pre-existing sqlite storage file which saved by montydb<=1.3.0 is not read/writeable after montydb==2.0.0. from montydb import set_storage, MontyClient set_storage("/db/repo", storage="sqlite") # required, to set sqlite as engine client = MontyClient("/db/repo") # ready to go SQLite config: [sqlite] journal_mode: WAL SQLite write concern: client = MontyClient("/db/repo", synchronous=1, automatic_index=False, busy_timeout=5000) LMDB (Lightning Memory-Mapped Database) lightning is NOT the default on-disk storage, need configuration first before get client. Newly implemented. from montydb import set_storage, MontyClient set_storage("/db/repo", storage="lightning") # required, to set lightning as engine client = MontyClient("/db/repo") # ready to go LMDB config: [lightning] map_size: 10485760 # Maximum size database may grow to. URI Optionally, You could prefix the repository path with montydb URI scheme. client = MontyClient("montydb:///db/repo") Utilities Pymongo bson may required. * montyimport Imports content from an Extended JSON file into a MontyCollection instance. The JSON file could be generated from montyexport or mongoexport. from montydb import open_repo, utils with open_repo("foo/bar"): utils.montyimport("db", "col", "/path/dump.json") * montyexport Produces a JSON export of data stored in a MontyCollection instance. The JSON file could be loaded by montyimport or mongoimport. from montydb import open_repo, utils with open_repo("foo/bar"): utils.montyexport("db", "col", "/data/dump.json") * montyrestore Loads a binary database dump into a MontyCollection instance. The BSON file could be generated from montydump or mongodump. from montydb import open_repo, utils with open_repo("foo/bar"): utils.montyrestore("db", "col", "/path/dump.bson") * montydump Creates a binary export from a MontyCollection instance. The BSON file could be loaded by montyrestore or mongorestore. from montydb import open_repo, utils with open_repo("foo/bar"): utils.montydump("db", "col", "/data/dump.bson") * MongoQueryRecorder Record MongoDB query results in a period of time. Requires to access databse profiler. This works via filtering the database profile data and reproduce the queries of find and distinct commands. from pymongo import MongoClient from montydb.utils import MongoQueryRecorder client = MongoClient() recorder = MongoQueryRecorder(client["mydb"]) recorder.start() # Make some queries or run the App... recorder.stop() recorder.extract() {: [, , ...], ...} * MontyList Experimental, a subclass of list, combined the common CRUD methods from Mongo's Collection and Cursor. from montydb.utils import MontyList mtl = MontyList([1, 2, {"a": 1}, {"a": 5}, {"a": 8}]) mtl.find({"a": {"$gt": 3}}) MontyList([{'a': 5}, {'a': 8}]) Why I did this ? Mainly for personal skill practicing and fun. I work in VFX industry, some of my production needs (mostly edge-case) requires to run in a limited environment (e.g. outsourced render farms), which may have problem to run or connect a MongoDB instance. And I found this project really helps. --------------------------------------------------------------------- drawing About Monty, Mongo tinified. MongoDB implemented in Python ! Topics python mongodb lmdb pymongo mocking tinydb Resources Readme License BSD-3-Clause License Releases 9 2.3.5 Latest May 29, 2021 + 8 releases Packages 0 No packages published Contributors 2 * @davidlatwe davidlatwe David Lai * @jqueguiner jqueguiner Jean-Louis Queguiner Languages * Python 100.0% * (c) 2021 GitHub, Inc. * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.