Welcome to
PythonCGI.net

This page is to explain how to use CGI, common gateway interface, to develop web pages in the Python3 programming language.

You will be able to run Python3 cgi .py programs similar to how PHP .php programs are run from Apache2.

The Apache2 Server is used to serve Python3 web pages via cgi.


Let's begin...

1.0 Configure Apache2 for Python3 CGI

Note: you may need to put sudo before commands on this page.

The command to enable cgi in Apache2 to serve Python3 web pages is the following:

a2enmod cgi

Then restart Apache2 with the command:

systemctl restart apache2

so the Apache2 changes can take effect.

1.1 Modify Apache2 VirtualHost to allow Python3 files to run/execute

In the file /etc/apache2/sites-enabled/000-default.conf

<VirtualHost *:80>
	ServerName www.awesomewebsite.com
	ServerAlias awesomewebsite.com *.awesomewebsite.com
    
	ServerAdmin webmaster@localhost
	DocumentRoot /var/awesomewebsite/www/AwesomeWebsite
	DirectoryIndex index.py
	ErrorLog ${{APACHE_LOG_DIR}}/error.log
	CustomLog ${{APACHE_LOG_DIR}}/access.log combined
	
	<Directory "/var/awesomewebsite/www/AwesomeWebsite">
        Options +ExecCGI -Indexes
        AddHandler cgi-script .py
        AddHandler default-handler .jpg .png .gif .css .js .ico
        AllowOverride None
        Require all granted	
	</Directory>
</VirtualHost>

This virtualhost code block for Apache 2.4 will allow Python3 programs to run under the DocumentRoot folder.

The +ExecCGI flag tells Apache2 to run a Python3 program, note that the Python3 program also needs its run/execute permission set on Linux.

And on Linux please use unix line endings in the Python3 source code text file, otherwise it may not work.

Notepad++ for example can display and even change the text file line endings easily under Edit -> EOL conversions.

2.0 Write a first Python3 CGI program

1.   #!/usr/bin/python3

2.   import os
3.   import sys

4.   import cgi
5.   import cgitb

6.   include_path = '/var/awesomewebsite/www'
7.   cgitb.enable(display=0, logdir=f'''{{include_path}}/tmp_errors''')
8.   sys.path.insert(0, include_path)

9.   from html import escape

10.  def enc_print(string='', encoding='utf8'):
11.      sys.stdout.buffer.write(string.encode(encoding) + b'\n')
	
12.  html = f'''

13.  <html>

14.  <head>

15.  <meta charset="UTF-8">

16.  </head>

17.  <body>

18.  <h1> Hello World 

19.  <p> I am outputting HTML from a Python3 CGI program. </p>

20.  </body>

21.  </html>

22.  '''   
    
23.  if __name__ == '__main__':
24.      enc_print("Content-Type:text/html;charset=utf-8;")
25.      enc_print()
26.      enc_print(html)

You may save this program without the line numbers as index.py under the /var/awesomewebsite/www/AwesomeWebsite folder.

Line 1. indicates that this is a Python3 program on Linux.

Lines 2., 3. imports modules that may be used.

Lines 4., 5. imports the important cgi and cgitb module. The cgi module will allow us for example to receive arguments/data from a html get or post. The cgitb is very important because it will help with outputting error information when an error has occurred. Though this error file that is stored in the tmp_errors folder is not the only place to identify program errors, the /var/log/apache2 will also contain the file error.log and this is the second place you may identify problems in your Python3 cgi program. Any good web developer using Python3 cgi knows that the error logs are our helpers that assist us immensely and boost our Python3 cgi web programming productivity to the next level. The third way to identify errors, mostly syntax errors, is to run the Python3 cgi program from the command line and see what the Python3 interpretor says.

Lines 6., 7., 8. set the program on where error output goes, just note that sometimes syntax errors will not be in this folder because the program cannot run to generate the error output in the folder /var/awesomewebsite/www/tmp_errors

Line 9. is an important line to import the module that will allow us to sanitize the html get and post variables from malicious data.

Lines 10., 11. are very important so that our Python3 cgi web program can be modern by outputting and encoding to unicode for emojis and foreign languages.
Or as of Python3.7
You can reconfigure the encoding of stdout as of python 3.7
    import sys
    sys.stdout.reconfigure(encoding='utf-8')            # Preferred way to display utf-8

    Then simply use print() instead of enc_print() statement.

Lines 12., to 22. create a Python3 variable to output the html that will display on the web browser.

Line 23. is the Python3 way to indicate a Python3 program, it's optional, though recommended at least in the index.py file. Other Python3 cgi programs it's surely is optional.

Lines 24., 25., 26. are the statements that output the python3 variable html in unicode format.

Congratulations, if your Python3 cgi web program runs successfully, you are ready to now improve the program to meet your needs and specifications for great things!

3.0 How to include another Python3 script in your main program

with open('myfile.py') as f: exec(f.read())

Way #1

def run_file(path):
    return exec(open(path).read())
    
run_file('myfile.py')

Way #2
    with open(f'''{{include_path}}/myfile.py''') as f: exec(f.read())


Way #3
    import importlib.util

    sp = importlib.util.spec_from_file_location(module_name, file_path)
    module = importlib.util.module_from_spec(s)
    sp.loader.exec_module(module)

4.0 How to sanitize html get or post form data

args = cgi.FieldStorage()
cars = '' if not args.getvalue('cars') else escape( args.getvalue('cars') )


For example, receiving data from a form with the variable cars.

The Python3 variable cars will either be the empty string or if there is a data in the variable it will be sanitized by the function escape()


5.0 How to Create Python Simple Routes With Using a Simple Apache2 Rewrite Engine Statement

First, configure Apache2
    In the file /etc/apache2/sites-enabled/000-default.conf
    
    <Directory "/var/awesomewebsite/www/AwesomeWebsite">
		RewriteEngine On
		RewriteRule (.*) index.py [L]
    </Directory>

Second, create a function mapper to split on the items in the url 

example: https://AwesomeWebsite.com/company/product_name/product_id

ARGS = {}

def mapper(e):
    global ARGS
    #os.environ["REQUEST_URI"]
    
    v = os.environ["REQUEST_URI"].split('/')
    v_len = len(v)
    if v_len == 4:
        ARGS['company'] = v[1]
        ARGS['product_name'] = v[2]
        ARGS['product_id'] = v[3]
    
( change mapper function as needed )


6.0 How to Redirect the Output of stdout into a String Buffer of an Included Python Script In Your Main Program

import os
from io import StringIO
from contextlib import redirect_stdout, redirect_stderr

directory = os.path.dirname(os.environ["SCRIPT_FILENAME"])

def connect(file, buffer_store=False):
        
    if buffer_store == True:
        with io.StringIO() as buf, redirect_stdout(buf):
            print('Redirected to string buffer')
            with open(directory + file) as f: 
                exec(f.read())
            return buf.getvalue()
    else:
        print('Direct output, no buffering')
        with open(directory + file) as f:                         
            exec(f.read())

7.0 How to Redirect the stderr to a file

So for programming, whenever there is an error, the import cgi and import cgitb and the cgitb.enable() statements are good.

However, you don't want to display these types of errors to the user.  Therefore it's much better to redirect stderr to a file.

This can be done with the following code:

import logging
from datetime import datetime

log_filename = '/var/awesomewebsite/www/tmp_errors/' + datetime.now().strftime("%Y-%m-%d %H-%M-%S")
log_formatter = logging.Formatter('%(asctime)s | %(levelname)s | %(message)s')
log = logging.getLogger()
fh = logging.FileHandler(filename=log_filename, mode='w', delay=True)  
fh.setFormatter(log_formatter)
log.addHandler(fh)

def show404():
    return '<html><body><h1>404</h1></body></html>'

def exception_handler(exc_type, exc_value, exc_traceback):
    if issubclass(exc_type, KeyboardInterrupt):
        # Let the system handle things like CTRL+C
        sys.__excepthook__(*args)
    log.error('Exception: ', exc_info=(exc_type, exc_value, exc_traceback))
    if os.path.exists(log_filename):
        print(show404(''))

def init():
    sys.excepthook = exception_handler


usage:
init()

8.0 How to replace cgi module that is depreciated in 3.13 version of Python

Option 1)
Simply install the pip package from https://pypi.org/project/legacy-cgi/

and done.

Option 2) 
install the pip package multipart from https://github.com/defnull/multipart

pip install multipart

Note:

modify multipart.py line 15 to include MultiDict as so:
__all__ = ["MultiDict", "MultipartError", "MultipartParser", "MultipartPart", "parse_form_data"]

#!/usr/bin/python3 # The MIT License # Copyright 2023 BestInternetSearch.com Inc. # Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: # The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. # THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. import os import sys from html import escape import urllib from multipart import MultipartParser, MultiDict, parse_options_header forms, files = MultiDict(), MultiDict() def __init__(self, os_environ, escape=True): self._os_environ = os_environ self._escape = escape def FieldStorage(self): e = self._os_environ if 'REQUEST_METHOD' in e: if e['REQUEST_METHOD'] == 'GET': self._data = urllib.parse.parse_qs(e['QUERY_STRING']) elif e['REQUEST_METHOD'] == 'POST': if e['QUERY_STRING'] != '': self._data = urllib.parse.parse_qs(e['QUERY_STRING']) if 'CONTENT_TYPE' in e: content_type, options = parse_options_header(e['CONTENT_TYPE']) if 'multipart/form-data' in e['CONTENT_TYPE']: if 'boundary' in e['CONTENT_TYPE']: boundary = options.get("boundary", "") if 'CONTENT_LENGTH' in e: content_length = int(os.environ.get("CONTENT_LENGTH", "-1")) stream = sys.stdin.buffer try: for part in MultipartParser(stream, boundary, content_length): if part.filename or not part.is_buffered(): files[part.name] = part except Exception as err: print(f"{type(err).__name__} exception: {err}") if 'application/x-www-form-urlencoded' in e['CONTENT_TYPE']: self._data = urllib.parse.parse_qs(str(sys.stdin.read()), keep_blank_values=True) if e['QUERY_STRING'] != '': self._data.update(urllib.parse.parse_qs(e['QUERY_STRING'])) def getvalue(self, arg_key, default=''): if arg_key in self._data: value = self._data[arg_key] if isinstance(value, list): if self._escape == True: return escape(self._data[arg_key][0]) else: self._data[arg_key][0] else: if self._escape == True: return escape(value) else: return value else: return default usage: mycgi = MyCGI(os.environ) mycgi.FieldStorage() user = mycgi.getvalue('user') And uploading files something like: curl -F "text=default" -F "filename=@/home/computer/Desktop/aa.png" -F "filename=@/home/computer/Desktop/bb.png" http://example_website.com filenames = files.getall('filename') for x, file in enumerate(filenames): file.save_as(f"""/home/computer/Desktop/file{x}.png""") # to print POST or GET parameters for key, values in mycgi._data.items(): for value in values: mycgi._data[key] = value print (f"""key:{key} value:{value}<br>""") # POST # curl -d "param1=value1&param2=value2" -X POST http://example_website.com # GET # curl http://example_website.com?user=user1

2024 improvement

Python3 neat function that I use in my projects:
def include(str_fullpath_page):
    with open(str_fullpath_page) as f: exec(f.read(), globals())
Note: still to code when upload post is: "application/x-url-encoded" email if you want with the code. This page is created by Stan Switaj, I am the CEO of UltraHot.TV Inc., creator/owner of HotStuff mobile app, CEO of BestInternetSearch.com Inc., CEO of GitBusiness.com Inc. Thanks for being a Python3 cgi web programming fan. Email me: opensource3 at yahoo dot com

Display python3 information with pyinfo like php's phpinfo

To display information like php's phpinfo() does in python3, use the following code.
Download the following python code pyinfo include it in your program, and run pyinfo() within a python3 cgi website.


There is always WSGI to serve a Python website. Though cgi has been very stable and works nicely also.