https://github.com/galenmarchetti/jupyter-notebook-package Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} galenmarchetti / jupyter-notebook-package Public * Notifications * Fork 1 * Star 38 A Kurtosis package for Python data engineers, deploying a Jupyter notebook along with a configurable set of databases, and a visualization tool (Streamlit) License Apache-2.0 license 38 stars 1 fork Activity Star Notifications * Code * Issues 0 * Pull requests 0 * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Security * Insights galenmarchetti/jupyter-notebook-package This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 3 branches 0 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/g] Use Git or checkout with SVN using the web URL. [gh repo clone galenm] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @galenmarchetti galenmarchetti Update README.md ... 8f65e60 Oct 27, 2023 Update README.md 8f65e60 Git stats * 65 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time jupyter merging main October 26, 2023 10:11 streamlit_app mongodb October 26, 2023 09:53 .gitignore gitignore October 25, 2023 23:41 LICENSE Initial commit October 8, 2023 19:27 README.md Update README.md October 27, 2023 16:13 config.json.tmpl adding mongodb hookup October 26, 2023 09:24 kurtosis.yml kurtosis.yml October 25, 2023 12:48 main.star merging main October 26, 2023 10:11 startup.py.tmpl adding mongodb hookup October 26, 2023 09:24 View code Jupyter Notebook + Database + Streamlit App Running the Environment Prototyping your Data App Crunching data in Jupyter Viewing the Streamlit app frontend Editing the Streamlit app Making Changes README.md Jupyter Notebook + Database + Streamlit App This is a free, local prototyping tool for Python developers crunching data and making visualizations. It connects Jupyter, a database of your choice, and Streamlit seamlessly for you, so you can use Jupyter to prototype your data gathering and Streamlit to prototype your data visualizations. There are two main reasons to spin up Jupyter+DB+Streamlit this way: * You automatically get pre-loaded PyMongo or SQLAlchemy clients in your Jupyter environment, with connection URLs correctly configured to your database. You also get the same in the Streamlit app. * It's a one-line deploy from this Github locator (github.com/ galenmarchetti/jupyter-notebook-package), so not much else to think about in order to get started jupyter-db-streamlit-15mb-shorter Specifically, this is a Kurtosis package that deploys: * A Jupyter notebook with pre-loaded SqlAlchemy/PyMongo clients, hooked into * A database (your choice of Postgres, MongoDB, or both) * A basic Streamlit App with pre-loaded database clients, automatically connected to the databases you chose to deploy The architecture of the system on your laptop, running over Docker, will look like: jupyter-database-package-diagram To use this prototyping tool, you just need to install Kurtosis and its dependencies (listed in the install guide). Running the Environment 1. Start with Postgres and MongoDB (default) kurtosis run github.com/galenmarchetti/jupyter-notebook-package 2. Start with just Postgres kurtosis run github.com/galenmarchetti/jupyter-notebook-package '{"mongodb_enabled": false}' 3. Start with just MongoDB kurtosis run github.com/galenmarchetti/jupyter-notebook-package '{"postgres_enabled": false}' Prototyping your Data App Crunching data in Jupyter * Go to the "notebook" URL in the output to enter the Jupyter notebook. + The password by default is kurtosis. + Here you can mess around with pulling data from APIs, scraping websites, and putting the results into either Postgres or MongoDB. + You can use !pip install in the notebook to install more Python packages. notebook-circled-output Viewing the Streamlit app frontend * Go to the "app-frontend" URL in the output to see the Streamlit app frontend smaller-app-frontend-circled Editing the Streamlit app * To work on the Streamlit app, there's two ways to do it: your own IDE (slower iteration loop, but your own settings), or the pre-installed VSCode IDE (faster iteration loop, but a standard vanilla VSCode installation). + Pre-installed VSCode IDE: Click on the "vscode" URL in the output to open the VSCode IDE, which will modify your python files on disk. smaller-vs-code-interface-circled * Your own IDE: Clone this repository, cd into it, and instead of running kurtosis run github.com/galenmarchetti/ jupyter-notebook-package , run the following: kurtosis run . Then, you can change your Python code using your IDE of choice, pointing it to streamlit_app/ within this repository. Once you're done making your changes, you can re-run the above kurtosis run . command to create a new enclave with your changes loaded into the Streamlit service. Making Changes You can make issues or submit PRs to this repository if you want to make changes, but I recommend you just fork it and take it where you want to take it. The repository defines a Kurtosis package and you can go to the Kurtosis docs for information on how to modify it to make it do what you need! About A Kurtosis package for Python data engineers, deploying a Jupyter notebook along with a configurable set of databases, and a visualization tool (Streamlit) Topics python data-science jupyter jupyter-notebook streamlit kurtosis-package Resources Readme License Apache-2.0 license Activity Stars 38 stars Watchers 1 watching Forks 1 fork Report repository Releases No releases published Packages 0 No packages published Contributors 2 * @galenmarchetti galenmarchetti Galen Marchetti * @h4ck3rk3y h4ck3rk3y Gyanendra Mishra Languages * Starlark 54.6% * Jupyter Notebook 26.1% * Python 15.0% * Dockerfile 4.3% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time.