If you are working with a Python project that extends beyond a few simple scripts and data files, you will know the headaches that can be generated as the project grows. This can result in a cluttered folder consisting of input files, intermediate files mixed together with several Python files and / or notebook files, and even project documentation. This can make it hard to find the data or the functions you need when you are looking for it.

There are numerous articles out there detailing how to structure Python projects, which are very well-detailed. However, when it comes to Streamlit (one of my favourite Python tools for developing web-based apps very quickly) I had trouble finding information on how best to structure a Streamlit app.

As a result, I have put together this article to suggest one possible way of structuring your Streamlit app to prevent it from becoming a tangled mess of files.

The Streamlit App folder structure and the Streamlit cookiecutter template proposed here are not the only ways an app could be structured. There are numerous other factors to consider, such as the type of app you are building and who that app is for. I encourage you to use the suggestions in this article for your own app and adapt it to suit your needs.

The Starting Point of Creating a Streamlit App

When I am starting with Streamlit — especially when I don’t fully know what I want to achieve or if I am experimenting — I will often start with a simple directory and a single app.py file.

streamlit_app
└── app.py

This is great, as we can put all of our code into app.py and then call upon the command streamlit run app.py to start the app in the browser.

This simple folder and single file can suddenly extend to multiple files as data is gathered from the relevant sources or generated from the main app.py file.

streamlit_app
├── app.py
├── data1.csv
├── random_data_file.csv
└── output.csv

However, there will come a point when the code within app.py starts to become unwieldy, messy and disorganised, and the application folder becomes filled with multiple data files with varying names.

Creating a Streamlit Project Folder Structure

Separating Input and Output Data Locations

Once the app has reached the point where it is no longer a one-off experiment, I will begin adding some structure. Not only does this help with keeping a neat workspace, but it also helps my sanity when finding files later!

The first step I normally do within the main app directory is to create subdirectories for input data and output data. These will be used to store any files that will be needed for the running of the app and any files generated as a result of any analysis.

streamlit_app/
├── data/
│ ├── data1.csv
│ └── random_data_file.csv
├── output/
│ └── results.csv
└── app.py

Adding Multiple Pages to a Streamlit App

When building web-based apps with Streamlit, it is very easy to continuously add sections and functions to app.py one after the other.

Very quickly, this leads to a very large and hard-to-navigate Python file.

One way to counter this is to split the information into multiple pages. This improves the look and feel of the app, as well as improving the user experience.

streamlit_app/
├── data/
│ ├── data1.csv
│ └── random_data_file.csv
├── output/
│ └── output.csv
├── pages/
│ ├── Page_1.py
│ └── Page_2.py
└── app.py

Streamlit has a nice way to handle multiple pages and that is by creating a pages subfolder and placing the .py files in there. I have previously covered this in the following article: https://towardsdatascience.com/creating-true-multi-page-streamlit-apps-the-new-way-2022-b859b3ea2a15

Creating Components For Reusability

Once multiple pages have been created, there may be reusable sections of code that you want to use on more than one page. One way to handle this is to create components. These can include special graphs, widgets and even the sidebar containing key application settings, which can easily be called from multiple places within the app.

streamlit_app/
├── data/
│ ├── data1.csv
│ └── random_data_file.csv
├── output/
│ └── output.csv
├── pages/
│ ├── Page_1.py
│ └── Page_2.py
├── src/
│ └── components/
│ ├── sidebar.py
│ └── special_graph_widget.py
└── app.py

Splitting Functional Code from UI Code

Finally, to reduce the size of app.py the key functionality can be extracted (which could include equations, data loading or graph generation) into a source (src) subfolder. This allows app.py and any pages to contain only code that will be used to generate key UI functionality and display.

streamlit_app/
├── data/
│ ├── data1.csv
│ └── random_data_file.csv
├── output/
│ └── output.csv
├── pages/
│ ├── Page_1.py
│ └── Page_2.py
├── src/
│ ├── components/
│ │ ├── sidebar.py
│ │ └── special_graph_widget.py
│ └── calculations/
│ ├── simple_maths.py
│ └── trig_functions.py
└── app.py

Adding Tests

When developing your code or application, you need to ensure that it works as expected, especially if you modify the function.

One way to ensure that any changes you make to a function or any other functions that it depends on is to write tests around that code.

There are several Python testing frameworks, such as PyTest and unittest, that are easy to use and even automate as part of a continuous integration workflow.

streamlit_app/
├── data/
│ ├── data1.csv
│ └── random_data_file.csv
├── output/
│ └── output.csv
├── pages/
│ ├── Page_1.py
│ └── Page_2.py
├── src/
│ ├── components/
│ │ ├── sidebar.py
│ │ └── special_graph_widget.py
│ └── calculations/
│ ├── simple_maths.py
│ └── trig_functions.py
├── tests/
│ └── calculations/
│ ├── test_simple_maths.py
│ └── test_trig_functions
└── app.py

Extra Folders: Images, Assets and CSS

As your Streamlit progresses, there may be a need to include company logos and other relevant images on your pages. These can easily be separated into an assets subfolder and further subdivided into whether they are images or custom css.

streamlit_app/
├── assets/
│ ├── css/
│ │ └── custom_style.css
│ └── images/
│ ├── logo.png
│ └── header.png
├── data/
│ ├── data1.csv
│ └── random_data_file.csv
├── output/
│ └── output.csv
├── pages/
│ ├── Page_1.py
│ └── Page_2.py
├── src/
│ ├── components/
│ │ ├── sidebar.py
│ │ └── special_graph_widget.py
│ └── calculations/
│ ├── simple_maths.py
│ └── trig_functions.py
├── tests/
│ ├── test_simple_maths.py
│ └── test_trig_functions
└── app.py

Cookiecutter

Creating the above structure manually each time you want to create a new Streamlit App can be tedious and time-consuming — meaning you are more likely to end up with a disorganised folder.

To help make the process of creating a Streamlit app simpler, we can use a library called cookiecutter. This library allows you to easily create projects based on templates — of which there are many available out there which range from Django apps to standard Python libraries.

By using cookiecutter, you can recreate a template that includes all of the directories and files mentioned above, including: data/, output/, pages/, src/, and even tests/. This can be further modified to suit your needs or the requirements for your project.

Once the template is setup, and you are ready to start a new Streamlit project, it’s a simple process of calling upon a single command in your terminal:cookiecutter location/of/the/template

This will populate your project with all the necessary folders and files for your new Streamlit app.

Streamlit Cookiecutter Template

Whilst learning about cookiecutter, I thought it would be handy to create my own template, which recreates the structure above. You can access it below, and I am more than happy for people to contribute to and expand.

To use the template, first ensure you have streamlit and cookie-cutter installed.

Then you open up a terminal or command prompt and type the following:

cookiecutter https://github.com/andymcdgeo/cookiecutter-streamlit.git

This will then ask you a series of questions, which will help determine which folders and files will be generated.

Please note this is still a work in progress, and I welcome any feedback to help improve the template.

Creating a Streamlit App Folder Structure Through MacOS Terminal

If you are working on a Mac, it is possible to do this directly in the terminal using the lines listed below. This will create a series of directories and subdirectories that closely mimic the above folder and file structure.

These could be stored within your note-taking app and pasted into a terminal window when needed.

This could also be converted into a bash script to help further automate the process.

mkdir -p streamlit_app/{data,output,pages,src/{components,calculations},tests/calculations}
touch streamlit_app/data/data1.csv streamlit_app/data/random_data_file.csv
touch streamlit_app/output/output.csv
touch streamlit_app/pages/Page_1.py streamlit_app/pages/Page_2.py
touch streamlit_app/src/components/sidebar.py streamlit_app/src/components/special_graph_widget.py
touch streamlit_app/src/calculations/simple_maths.py streamlit_app/src/calculations/trig_functions.py
touch streamlit_app/tests/calculations/test_simple_maths.py streamlit_app/tests/calculations/test_trig_functions.py
touch streamlit_app/app.py

Summary

Ensuring your Streamlit app is well organised can go a long way to helping you stay sane when developing your app or provide a nice starting point that saves you time by not having to create a new folder structure from scratch. Using cookiecutter templates, like the Streamlit Cookiecutter template can help automate the process and get you off to a better start when creating your app.

If you like the cookiecutter template or have suggestions for improvements, please feel free to leave a comment or a new case on the GitHub repo.