Script to automatically make FOIA requests
Project Lead:
Michael duPont (@mdupont)
This project is supposed to automate FOIA (Freedom of Information Act) requests that occur on on a regular basis. In addition to simplifying the requester's job, it has the added benefit of surfacing particular datasets for inclusion into open data portals. For this reason, it is recommended that the request include a line about placing the info into the open data portal. The hope is that these datasets become regularly updated in the portal without a FOIA prompt.
This project uses Python running inside an AWS Lambda function. Production settings are located in S3 while test settings are local.
This script works by making POST requests to NextRequest based on config data from a JSON file. The JSON file follows this template:
[
{
"config": {
"frequency": "[int][D/W]",
"last": "2017-01-01T00:00:00.000000" or null,
"enabled": true / false
},
"request": {
"request_text": "Message including desired data and handling"
},
"requester": {
"address": "123 Foo St",
"city": "Orlando",
"company": "Code for Orlando",
"email": "foo@bar.com",
"name": "Foo Bar",
"phone_number": "1234567890",
"state": "Florida",
"zipcode": "12345"
}
},
{...}
]
JSON Configs:
There is also a global TESTING flag at the top of the Python script to negate all enabled flags when True. Code will read config data locally, execute everything except the POST request, and act as though it was successful.
git clone https://github.com/cforlando/FOIA-Auto-Requester.git
cd FOIA-Auto-Requester
pip install -r requirements.txt
Double check that TESTING is set to True before running locally. If TESTING is set to False, the script should fail because it lacks AWS credentials to fetch the S3 configs.
python foiareq.py
There are three things we need to in order to deploy to AWS Lambda:
Downgrade script from Python3 to Python2. In its current state, the only things needed are to remove function annotations and change dict.items()
to dict.iteritems()
Download the dependencies. Because Lambda is like its own virtual env, any third-party libraries must be included with the code
pip2 install -r requirements.txt -t deployment/
Once that is done, you upload the zip file to AWS Lambda and check that the daily trigger is still enabled.
You will also need to create an S3 Bucket to read the production config file from. The name of the bucket and file are stored as S3_BUCKET and S3_KEY at the top of the code.
It's possible to add new requests to requests.config.json without having to test the program yourself. Contact us and/or create issues with any questions, bugs, or to add your request to the production version. Use this GitHub issue as an example.
MIT LICENSE file.
Code for Orlando, a local Code for America brigade, brings the community together to improve Orlando through technology. We are a group of “civic hackers” from various disciplines who are committed to volunteering our talents to make a difference in the local community through technology. We unite to improve the way the community, visitors, and local government experience Orlando.