https://github.com/pyrustic/shared Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code + Explore + All features + Documentation + GitHub Skills + Changelog * Solutions + By Size + Enterprise + Teams + Compare all + By Solution + CI/CD & Automation + DevOps + DevSecOps + Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadMe Project GitHub community articles + Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} pyrustic / shared Public * Notifications * Fork 0 * Star 10 Triptych for data exchange and persistence pyrustic.github.io License MIT license 10 stars 0 forks Star Notifications * Code * Issues 0 * Pull requests 0 * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Security * Insights pyrustic/shared This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. master Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags 1 branch 19 tags Code * Clone HTTPS GitHub CLI [https://github.com/p] Use Git or checkout with SVN using the web URL. [gh repo clone pyrust] Work fast with our official CLI. Learn more. * Open with GitHub Desktop * Download ZIP Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @pyrustic pyrustic Update ... 1881ca5 Sep 8, 2022 Update 1881ca5 Git stats * 44 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .backstage Update Sep 8, 2022 .pyrustic/buildver Update Sep 8, 2022 docs/modules Update Sep 7, 2022 shared Update Sep 7, 2022 tests Update Jun 22, 2022 .gitignore First commit May 22, 2021 LICENSE Update Jan 18, 2022 MANIFEST.in Update Jun 1, 2021 README.md Update Sep 8, 2022 VERSION Update Sep 8, 2022 backstage.tasks Update Sep 7, 2022 pyproject.toml First commit May 22, 2021 setup.cfg Update Jul 3, 2022 setup.py First commit May 22, 2021 View code [ ] Pyrustic Shared Table of contents Overview Designed as a triptych Some characteristics Initialization Data access Command-line interface Document Initialization Use a conditional statement Set default data Data location Default directory Absolute pathname Temporary data Autosave Caching Readonly Clear data Convenience functions Recapitulation Dossier Example Binary data Check Clear data Recapitulation Database Example Command line interface Check the content Read the content of a specific entry Store binary data Store a collection Delete an entry Delete a dossier Miscellaneous Installation For the first time Upgrade Show information README.md License: MIT PyPI package version Downloads Cover Pyrustic Shared Data exchange and persistence This project is part of the Pyrustic Open Ecosystem. Installation . Latest . Modules Table of contents * Overview * Document * Dossier * Database * Command line interface * Miscellaneous * Installation Overview Shared is a Python package created to be the programmer's companion when it comes to storing application data, managing configuration files, caching data, and exchanging data with other programs. Although a lightweight package, Shared smoothly handles collections ( dict, list, set), binary data, and SQL queries. Designed as a triptych Shared's intuitive application programming interface is designed as a triptych. Thus, three classes with similar interfaces are created to cover the needs of data exchange and persistence: Document, Dossier, and Database. Class Relevance Document For individual access to Jesth and JSON files that are likely to be manually edited by a human. Dossier To store collections and binary data in a dossier without worrying about how they are actually saved. Database For an intuitive interaction with SQLite databases. Note: The Document class is not intended to be used directly. Instead, depending on the requirement, one will use the JsonDocument or JesthDocument class which subclasses the Document class. Some characteristics Since all three classes share similar interfaces, some handy functionality has been replicated in all of them with a few exceptions. Initialization All three classes emphasize initialization: * Document and Dossier give the possibility to define default data. * Database allows the definition of an initialization SQL script which is only executed to create a new database. Data access All three classes provide an optional read-only access to data and also allow the creation of temporary data which is automatically deleted when the user closes the application. Document and Dossier provide Autosave functionality, while Database automatically closes the underlying database connection when the user closes the application. Command-line interface Among the three classes, Dossier is the class of which a single instance can handle multiple underlying files. The Dossier class has its own protocol for organizing data. For this reason, Dossier offers a simple yet powerful command-line interface that allows other programs or a human to read and write the contents of a dossier. Let's explore the Document, Dossier, and Database classes in the next sections ! Document The Document class represents an interface for reading and writing an underlying file whose format is either Jesth or JSON. As stated previously in the Overview section, the Document class is not intended to be used directly. Instead, depending on the requirement, one will use the JsonDocument or JesthDocument class which subclasses the Document class. Since JSON is very popular, we will focus on the JsonDocument class in the following examples. Accessing a document or creating a new one is as simple as this: from shared import JsonDocument # Create a new document instance which will be linked to the 'my-data.json' file. # If this file doesn't exist yet, it will be automatically created document = JsonDocument("my-data.json") # From now, we can use 'document' to read and write the contents of 'my-data.json' ! # ... The string my-data.json is the base name of a file that will be created if it does not yet exist. This string is called Target and can be an absolute path or an instance of pathlib.Path. The Document class exposes the read and write methods, respectively, to read and write the underlying document. Initialization A document can be initialized with a conditional statement or by defining default data. By default, the Document class will assign a dict to the null parameter default_data. Use a conditional statement It's as simple as testing a boolean to check if the underlying document file is newly created or not: from shared import JsonDocument # access 'my-data.json' document = JsonDocument("my-data.json") # let's initialize the content of 'my-data.json' if document.new: data = {"name": "alex", "job": "evangelist"} document.write(data) # persisted ! Set default data The most elegant, less verbose and recommended way to initialize a document is to set some default data to the default parameter: from shared import JsonDocument # default data to init the file 'my-data.json' DEFAULT_DATA = {"name": "alex", "job": "evangelist"} # access 'my-data.json' document = JsonDocument("my-data.json", default=DEFAULT_DATA) # From now, thanks to the initialization functionality, the underlying # document contains the default data, assuming that 'my-data.json' # did not exist before the `Document` class was instantiated Data location The only mandatory argument to be supplied to the Document class constructor is the target. For convenience, the target is either the absolute path or the base name of a file. Its data type is either a string or an instance of pathlib.Path. The optional directory parameter exists to supplement the target value when that value is not an absolute path. Default directory By default, document files are saved in $HOME/PyrusticHome/shared. You can change the location according to your needs: from shared import JsonDocument DIRECTORY = "/home/alex/private" # access 'my-data.json' document = JsonDocument("my-data.json", directory=DIRECTORY) # From now, you can access these properties: # document.name == "my-data.json" # document.directory == "/home/alex/private" # document.target == "my-data.json" Absolute pathname You can set an absolute path as the target. In this case, the Document class ignores the directory parameter. from shared import JsonDocument pathname = "/home/alex/private/my-data.json" # access 'my-data.json' document = JsonDocument(pathname) # From now, you can access these properties: # document.name == "my-data.json" # document.directory == "/home/alex/private" # document.target == "/home/alex/private/my-data.json" Temporary data Setting the temporary boolean can enable temporary mode, so a document can only be created and used while the application is running, and then safely deleted when the application closes: from shared import JsonDocument # access 'my-data.json' document = JsonDocument("my-data.json", temporary=True) # This document will be created in a temporary directory # then it will be safely deleted when the application closes # or when the developer explicitly calls the 'close' or 'delete' method The Document class uses tempfile.TemporaryDirectory to implement this functionality. Autosave Thanks to atexit module, Document can autosave content when the application is closed: import sys from shared import JsonDocument # access 'my-config.json' with `autosave` mode enabled document = JsonDocument("my-config.json", autosave=True, default=[]) # load the data data = document.read() # few lines of code later... data.append("batman") # data modified sys.exit() # data automatically saved ! Along with atexit module, the Document class also uses a caching mechanism to implement the autosave functionality. Caching By default, caching mode is enabled, so the user can access cached data through the cache property of an instance of the Document class: from shared import JsonDocument DEFAULT_DATA = {"name": "alex", "job": "evangelist"} # access 'my-config.json' document = JsonDocument("my-config.json", caching=True, default=DEFAULT_DATA) data = document.read() if data is document.cache: print("Same same !") Readonly Setting the readonly parameter to True prevents the running application from accidentally modifying the content of a document: from shared import JsonDocument # access 'my-data.json' document = JsonDocument("my-data.json", readonly=True) # when you set readonly to True, you can no longer edit the content ! # shared.ReadonlyError will be raised if you try to mess with a readonly document Clear data You can delete the underlying file of a document (assuming the file isn't in readonly mode): from shared import JsonDocument # access 'my-data.json' document = JsonDocument("my-data.json") # delete 'my-data.json' document.delete() if document.deleted: print("Successfully deleted !") Convenience functions Four convenience functions are available for the JsonDocument class (also for the JesthDocument class): from shared import json_create, json_readonly, json_write, json_autosave # quickly create a document DEFAULT = ["red", "violet"] json_create("my-data.json", default=DEFAULT) # quickly open a document in readonly mode data = json_readonly("my-data.json") # quickly change the content of a document data = ["red", "green"] json_write("my-data.json", data) # quickly read the content of a document in autosave mode data = json_autosave("my-data.json") data.append("blue") # data will be automatically saved on exit Recapitulation For individual access to Jesth and JSON files that are likely to be manually edited by a human, the Document class is the recommended interface. For more technical details about this class and the subclasses JesthDocument and JsonDocument, read the reference documentation. Dossier The Dossier class stores collections (list, dict, set) and binary data with a unified interface inside a dossier. Shared allows to read and write a dossier not only programmatically but also from the command line. This class shares a similar interface with the Document class. Thus, the Dossier class constructor has target, directory, autosave, readonly and temporary as parameters. These parameters are already covered in the Document class section. Under the hood, Dossier uses files and JSON to store data. Example Let's create a dossier with script_1.py: # script_1.py from shared import Dossier # data people = {"Jack": "male", "Jane": "female"} # dict planets = ["Mars", "Venus", "Jupiter"] # list colors = {"red", "green", "blue"} # set # let's persist the data in 'my-dossier' dossier = Dossier("my-dossier") dossier.set("people", people) # set the 'people' entry dossier.set("planets", planets) # set the 'planets' entry dossier.set("colors", colors) # set the 'colors' entry # Done ! The data is persisted ! From script_2.py, let's access the dossier created with script_1.py: # script_2.py from shared import Dossier # let's access the shared dossier dossier = Dossier("my-dossier") # get data from the shared dossier people = dossier.get("people") # get the 'people' entry planets = dossier.get("planets") # get the 'planets' entry' colors = dossier.get("colors") # get the 'colors' entry print(people) # output: {'Jack': 'male', 'Jane': 'female'} print(planets) # output: ['Mars', 'Venus', 'Jupiter'] print(set(colors)) # there is nothing called 'set' in JSON [1] # output: {'red', 'green', 'blue'} # [1] the value of 'colors' is this dictionary: # {'red': None, 'green': None, 'blue': None} Binary data You can store binary data with the same unified interface: # script_1.py from shared import Dossier dossier = Dossier("my-dossier") with open("/home/alex/selfie.png", "rb") as file: data = file.read() dossier.set("selfie", data) # set the 'selfie' entry # the 'set' method returns the path to the binary file that stores the binary entry The above code can also be expressed like this: # script_1.py import pathlib from shared import Dossier dossier = Dossier("my-dossier") path = pathlib.Path("/home/alex/selfie.png") dossier.set("selfie", path) # set the 'selfie' entry # the 'set' method returns the path to the binary file that stores the binary entry You can retrieve your binary data from another script: # script_2.py from shared import Dossier from shutil import copyfile dossier = Dossier("my-dossier") source_path = dossier.get("selfie") # get the filename of the 'selfie' bin entry destination_path = "/home/alex/new.png" # copy the content from source to destination copyfile(source_path, destination_path) Check Use the check method to check the contents of a dossier or a specific entry: from shared import Dossier dossier = Dossier("my-dossier") # check a specific entry info = dossier.check("entry") if info: # info is a 3-tuple (name, container, filename). # The name is simply the entry name. # The container is a string that represents the type of the entry. # containers: "dict", "list", "set", and "bin" # The filename is either the path to a JSON file or a binary file name, container, filename = info # check the contents of the dossier dossier_info = dossier.check() # returns a dict, keys are entries and values are 3-tuples for entry, info in dossier_info.items(): print("Entry:", info.name) # the entry name print("Container:", info.container) # 'dict', 'set', 'list', or 'bin' print("Filename:", info.filename) # the underlying file in which the data is stored print() Clear data You can decide to delete a specific entry, a group of entries, or the dossier: from shared import Dossier dossier = Dossier("my-dossier") # delete a specific entry dossier.delete("entry_1") # delete a group of entries dossier.delete("entry_2", "entry_3") # delete the dossier dossier.delete() # collections, binary data, and meta data are gone Recapitulation To store collections and binary data in a dossier without worrying about how they are actually saved, the Dossier class is the interface to use. For more technical details about this class, read its documentation. Database Intuitive interaction with SQLite databases. Example The following example shows how nice it is to work with the Database class: from shared import Database # Initialization script # This SQL script will create two tables: friends and projects INIT_SCRIPT = """\ CREATE TABLE friends (name TEXT PRIMARY KEY, age INTEGER NOT NULL); CREATE TABLE projects (name TEXT PRIMARY KEY, language TEXT NOT NULL); """ # If this database doesn't exist yet, # it will be created with the initialization script database = Database("my-database", init_script=INIT_SCRIPT) # This will only be executed once ! # So you can safely restart this script again and again... if database.new: # Populate this database sql = """INSERT INTO friends VALUES ("Jack", 20)""" database.edit(sql) # few lines of code later... # Populate this database sql = """INSERT INTO friends VALUES (?, ?)""" parameters = ("Jane", 21) database.edit(sql, param=parameters) # Read data sql = "SELECT * FROM friends" columns, data = database.query(sql) # returns a shared.dto.QueryResult namedtuple print(columns) # output: ['name', 'age'] print(data) # output: [('Jack', 20), ('Jane', 21)] For more technical details about this class, read its documentation. Command line interface Shared comes with an intuitive command line interface for the Dossier class. Type help in the command line interface to display a short manual. For the following subsections, assume we have a pre-populated dossier named my-dossier and located in /home/alex/dossiers. Check the content Check the contents of my-dossier or a specific entry: $ cd /home/alex/dossiers/my-dossier $ shared check - 'colors' set 56B - 'people' dict 44B - 'planets' list 42B $ shared check people 'people' dict 44B $ shared check colors 'colors' set 56B Read the content of a specific entry $ cd /home/alex/dossiers/my-dossier $ shared get people { "Jack": "male", "Jane": "female" } $ shared get planets [ "Mars", "Venus", "Jupiter" ] shared get colors { "red": null, "blue": null, "green": null } The output text is the exact JSON representation as stored in a file. So the output can be consumed as is by another program and deserialized with a JSON library. Note that the colors entry is a set but represented as a dict in JSON. Store binary data $ shared set selfie bin: '/home/alex/selfie.png' Entry successfully updated ! You can copy a binary entry into an arbitrary file from the command line: $ shared get selfie > '/home/alex/selfie-copy.png' Store a collection $ shared set countries list: '/home/alex/countries.json' Entry successfully updated ! $ shared set my_config dict: '/home/alex/app_config.json' Entry successfully updated ! Delete an entry $ shared del "selfie" Entry successfully deleted ! $ shared check selfie This entry doesn't exist. Delete a dossier Right-click on the folder with your mouse, then send it safely to the trash... ;) Miscellaneous In June 2022, Shared entered the top 5000 most downloaded Python packages on PyPI over 30 days for the first time. Installation Shared is cross platform and versions under 1.0.0 will be considered Beta at best. It should work on Python 3.5 or newer. For the first time $ pip install shared Upgrade $ pip install shared --upgrade --upgrade-strategy eager Show information $ pip show shared Back to top About Triptych for data exchange and persistence pyrustic.github.io Topics python cli json data library database binary persistence configuration collections shared pyrustic probed-dict probed-list probed-set shared-dict shared-list shared-set shared-bin jesth Resources Readme License MIT license Stars 10 stars Watchers 1 watching Forks 0 forks Releases 19 tags Packages 0 No packages published Languages * Python 100.0% Footer (c) 2022 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.