From: https://towardsdatascience.com/develop-and-sell-a-python-api-from-start-to-end-tutorial-9a038e433966 You can also read this article directly on Github (for better code formatting) I recently read a blog post about setting up your own API and selling it. I was quite inspired and wanted to test if it works. In just 5 days I was able to create an API from start to end. So I thought I share issues I came across, elaborate on concepts that the article was introducing, and provide a quick checklist to build something yourself. All of this by developing another API. Table of Contents
About this articleThis article can be considered as a tutorial and comprehension of other articles (listed in my “Inspiration” section). It paints a picture for developing a Python API from start to finish and provides help in more difficult areas like the setup with AWS and Rapidapi. I thought it will be useful for other people trying to do the same. I had some issues on the way, so I thought I share my approach. It is also a great way to build side projects and maybe even make some money. As the Table of content shows, it consists of 4 major parts, namely:
You will find all my code open sourced on Github: You will find the end result here on Rapidapi: If you found this article helpful let me know and/or buy the functionality on Rapidapi to show support. DisclaimerI am not associated with any of the services I use in this article. I do not consider myself an expert. If you have the feeling that I am missing important steps or neglected something, consider pointing it out in the comment section or get in touch with me. Also, always make sure to monitor your AWS costs to not pay for things you do not know about. I am always happy for constructive input and how to improve. Stack usedWe will use
1. Create project formalitiesIt’s always the same but necessary. I do it along with these steps:
git remote add origin URL_TO_GIT_REPO Now we have:
2. Create a solution for a problemThen we need to create a solution to some problem. For the sake of demonstration, I will show how to convert an excel csv file into other formats. The basic functionality will be coded and tested in a Jupyter Notebook first.
Install packagesInstall jupyter notebook and jupytext: pip install notebook jupytext sets a hook in #!/bin/shjupytext --from ipynb --to jupytext_conversion//py:light --pre-commit Develop a solution to a problempip install pandas requests Add a Download dataDownload an example dataset (titanic dataset) and save it into a data folder: def download(url: str, dest_folder: str): Create functionalityTransform format df = pd.read_csv('./data/titanic.csv') Build server to execute a function with RESTAfter developing the functionality in jupyter notebook we want to actually provide the functionality in a python app. There are ways to use parts of the jupyter notebook, but for the sake of simplicity we create it again now. Add an We want the user to upload an excel file and return the file converted into JSON for example. Browsing through the internet we can see that there are already packages that work with flask and excel formats. So let's use them. pip install Flask Start Flask server with env FLASK_APP=app.py FLASK_ENV=development flask run Tipp: Test your backend functionality with Postman. It is easy to set up and allows us to test the backend functionality quickly. Uploading an excel is done in the “form-data” tab: ![]() Here you can see the uploaded titanic csv file and the returned column names of the dataset. Now we simply write the function to transform the excel into json, like: import json (Check out my repository for the full code.) Now we have the functionality to transform csv files into json for example. 3. Deploy to AWSAfter developing it locally we want to get it in the cloud.
Set up zappaAfter we created the app locally we need to start setting up the hosting on a real server. We will use zappa.
pip install zappa As we are using a conda environment we need to specify it: which python will give you remove the export VIRTUAL_ENV=/Users/XXXX/opt/anaconda3/envs/XXXXX/ Now we can do zappa init to set up the config. Just click through everything and you will have a { Note that we are not yet ready to deploy. First, we need to get some AWS credentials. Set up AWSAWS credentialsFirst, you need te get an AWS You might think it is as easy as: To get the credentials you need to
But no. There is more to permissions in AWS! Set up credentials with users and roles in IAMI found this article from Peter Kazarinoff to be very helpful. He explains the next section in great detail. My following bullet point approach is a quick summary and I often quote his steps. Please check out his article for more details if you are stuck somewhere. I break it down as simple as possible:
My Custom policy: { NOTE: Replace XXXXXXXXXXX in the inline policy by your AWS Account Number. Your AWS Account Number can be found by clicking “Support → “Support Center. Your Account Number is listed in the Support Center on the upper left-hand side. The json above is what worked for me. But, I expect this set of security permissions may be too open. To increase security, you could slowly pare down the permissions and see if Zappa still deploys. The settings above are the ones that finally worked for me. You can dig through this discussion on GitHub if you want to learn more about specific AWS permissions needed to run Zappa: https://github.com/Miserlou/Zappa/issues/244. Add credentials in your projectCreate a mkdir ~/.aws and paste your credentials from AWS [dev] Same with the code open ~/.aws/config[default] Note that Save the AWS access key id and secret access key assigned to the user you created in the file ~/.aws/credentials. Note the .aws/ directory needs to be in your home directory and the credentials file has no file extension. Now you can do deploy your API with zappa deploy dev ![]() There shouldn’t be any errors anymore. However, if there are still some, you can debug with: zappa status The most common errors are permission related (then check your permission policy) or about python libraries that are incompatible. Either way, zappa will provide good enough error messages for debugging. If you update your code don’t forget to update the deployment as well with zappa update dev AWS API GatewayTo set up the API on a market we need to first restrict its usage with an API-key and then set it up on the market platform. I found this article from Nagesh Bansal to be helpful. He explains the next section in great detail. My following bullet point approach is a quick summary and I often quote his steps. Please check out his article for more details if you are stuck somewhere. Again, I break it down:
it looks like this ![]() Now you have restricted access to your API. 4. Set up Rapidapi
Create API on Rapidapi
![]() 5. In the security tab you can check everything 6. Then go to “endpoints” to add the routes from you Python app by clicking “create REST endpoint” ![]() 7. Add an image for your API 8. Set a pricing plan. Rapidapi published an own article on pricing options and strategies. As they conclude, it is up to your preferences and product on how to price it. 9. I created a freemium pricing plan. The reason for that is that I want to give the chance to test it without cost, but add a price for using it regularly. Also, I want to create a plan for supporting my work. For example: ![]() 10. Create some docs and a tutorial. This is pretty self-explaining. It is encouraged to do so as it is easier for people to use your API if it is documented properly. 11. The last step is to make your API publicly available. But before you do that it is useful to test it for yourself. Test your own APICreate a private plan for testingHaving set up everything, you of course should test it with the provided snippets. This step is not trivial and I had to contact the support to understand it. Now I am simplifying it here. Create a private plan for yourself, by setting no limits. The go to the “Users” section of your API, then to “Users on free plans”, select yourself and “invite” you to the private plan. ![]() ![]() Create code to consume APITo consume the API now you can simply copy the snippet that Rapidapi provides. For example with Python and the requests library: import requestsurl = "https://excel-to-other-formats.p.rapidapi.com/upload"payload = "" End resultInspirationThe article “API as a product. How to sell your work when all you know is a back-end” by Artem provided a great idea, namely to
For the setting everything I found the articles from Nagesh Bansal very helpful:
Also this article from Peter Kazarinoff: https://pythonforundergradengineers.com/deploy-serverless-web-app-aws-lambda-zappa.html I encourage you to have a look at those articles as well. You can also read my article directly on Github (for better code formatting) |
Python >