Skip to content

Commit 70d209c

Browse files
committed
refactor: reorganize documentation artifacts into separate folders
1 parent 4957f63 commit 70d209c

File tree

73 files changed

+478
-61
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

73 files changed

+478
-61
lines changed

README.md

+5-2
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,12 @@
77
[![ReactJS](https://a11ybadges.com/badge?logo=react)](https://react.dev/)
88
[![Python](https://a11ybadges.com/badge?logo=python)](https://www.python.org/)
99
[![Terraform](https://a11ybadges.com/badge?logo=terraform)](https://www.terraform.io/)<br>
10-
[![Release Notes](https://img.shields.io/github/release/FullStackWithLawrence/aws-openai)](https://github.com/FullStackWithLawrence/aws-openai/releases)
10+
[![12-Factor](https://img.shields.io/badge/12--Factor-Compliant-green.svg)](./doc/Twelve_Factor_Methodology.md)
11+
[![Unit Tests](https://github.com/FullStackWithLawrence/aws-openai/actions/workflows/tests.yml/badge.svg)](https://github.com/FullStackWithLawrence/aws-openai/actions)
1112
![GHA pushMain Status](https://img.shields.io/github/actions/workflow/status/FullStackWithLawrence/aws-openai/pushMain.yml?branch=main)
12-
[![AGPL License](https://img.shields.io/github/license/overhangio/tutor.svg?style=flat-square)](https://www.gnu.org/licenses/agpl-3.0.en.html)
13+
![Auto Assign](https://github.com/FullStackwithLawrence/aws-openai/actions/workflows/auto-assign.yml/badge.svg)
14+
[![Release Notes](https://img.shields.io/github/release/FullStackWithLawrence/aws-openai)](https://github.com/FullStackWithLawrence/aws-openai/releases)
15+
[![License: AGPL v3](https://img.shields.io/badge/License-AGPL_v3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)
1316
[![hack.d Lawrence McDaniel](https://img.shields.io/badge/hack.d-Lawrence%20McDaniel-orange.svg)](https://lawrencemcdaniel.com)
1417

1518
A [React](https://react.dev/) + [AWS Serverless](https://aws.amazon.com/serverless/) full stack implementation of the [30 example applications](https://platform.openai.com/examples) found in the official OpenAI API documentation. Now with [LangChain](https://www.langchain.com/)!

api/README.md

+5-5
Original file line numberDiff line numberDiff line change
@@ -188,11 +188,11 @@ Example valid request body:
188188
```
189189

190190
_Note the output variables for your API Gateway root URL and API key._
191-
![Postman](https://raw.githubusercontent.com/FullStackWithLawrence/aws-openai/main/doc/terraform-apply2.png "Postman")
191+
![Postman](https://raw.githubusercontent.com/FullStackWithLawrence/aws-openai/main/doc/img/terraform-apply2.png "Postman")
192192

193193
5. (Optional) use the [preconfigured import files](./postman/) to setup a Postman collection with all 30 URL endpoints.
194194

195-
![Postman](https://raw.githubusercontent.com/FullStackWithLawrence/aws-openai/main/doc/postman-1.png "Postman")
195+
![Postman](https://raw.githubusercontent.com/FullStackWithLawrence/aws-openai/main/doc/img/postman-1.png "Postman")
196196

197197
### Custom Domain (Optional)
198198

@@ -205,7 +205,7 @@ root_domain = "yourdomain.com"
205205

206206
## How It Works
207207

208-
![API Workflow](https://raw.githubusercontent.com/FullStackWithLawrence/aws-openai/main/doc/api-workflow.png "API Workflow")
208+
![API Workflow](https://raw.githubusercontent.com/FullStackWithLawrence/aws-openai/main/doc/img/api-workflow.png "API Workflow")
209209

210210
1. a JSON object and custom headers are added to an HTTP request and sent to the API as a 'PUT' method.
211211
2. API Gateway uses a [Request Mapping Template](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html) in a non-proxy Lambda integration request to combine user request text with your OpenAPI application definition, and then forward the combined data as a custom JSON object to a Lambda function.
@@ -332,8 +332,8 @@ The terraform scripts will automatically create a collection of CloudWatch Log G
332332

333333
I refined the contents and formatting of each log group to suit my own needs while building this solution, and in particular while coding the Python Lambda functions.
334334

335-
![CloudWatch Logs](https://raw.githubusercontent.com/FullStackWithLawrence/aws-openai/main/doc/cloudwatch-1.png "CloudWatch Logs")
336-
![CloudWatch Logs](https://raw.githubusercontent.com/FullStackWithLawrence/aws-openai/main/doc/cloudwatch-2.png "CloudWatch Logs")
335+
![CloudWatch Logs](https://raw.githubusercontent.com/FullStackWithLawrence/aws-openai/main/doc/img/cloudwatch-1.png "CloudWatch Logs")
336+
![CloudWatch Logs](https://raw.githubusercontent.com/FullStackWithLawrence/aws-openai/main/doc/img/cloudwatch-2.png "CloudWatch Logs")
337337

338338
## Documentation
339339

doc/CONTRIBUTING.md

+72
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# Contributing - Developer Setup Guide
2+
3+
This repository contains two distinct projects, respectively, written in
4+
5+
- [Python](#python-setup)
6+
- [Terraform](#terraform-setup)
7+
8+
In each case there are various technology-specific resources that you'll need to initialize in your development environment.
9+
10+
## Repository Setup
11+
12+
### GitHub Actions
13+
14+
As a 1-person operation this project depends heavily on GitHub Actions to automate routine activities, so that hopefully, the source code is always well-documented and easy to read, and everything works perfectly. We automate the following in this project:
15+
16+
- Code linting checks, during both pre-commit as well as triggered on pushes to the main branch
17+
- Unit tests for Python, React and Terraform
18+
- Semantic releases
19+
- version bumps from npm, PyPi and Terraform Registry
20+
21+
A typical pull request will look like the following:
22+
23+
![Automated pull request](https://github.com/FullStackWithLawrence/aws-openai/blob/main/doc/img/automated-pr.png)
24+
25+
### pre-commit setup
26+
27+
This project uses pre-commit as a first-pass automated code review / QC process. pre-commit runs a multitude of utilities and checks for code formatting, linting, syntax checking, and ensuring that you don't accidentally push something to GitHub which you'd later regret. Broadly speaking, these checks are aimed at minimizing the extent of commits that contain various kinds of defects and stylistic imperfections that don't belong on the main branch of the project.
28+
29+
Note that many of the pre-commit commands are actually executed by Python which in turn is calling pip-installed packages listed in requirements.txt located in the root of the repo. It therefore is important that you first create the Python virtual environment using `make init`. It also is a good idea to do a complete 'dry run' of pre-commit, to ensure that your developer environment is correctly setup:
30+
31+
```console
32+
git pull
33+
make pre-commit
34+
```
35+
36+
Output should look similar to the following:
37+
38+
![pre-commit output](https://github.com/FullStackWithLawrence/aws-openai/blob/main/doc/img/pre-commit.png)
39+
40+
### Github Secrets setup
41+
42+
The GitHub Actions automated processes depend on several credentials which are stored inside of Github Secrets. When creating pull requests, the GitHub Actions will use these secrets, [github.com/FullStackWithLawrence/aws-openai/settings/secrets/actions](https://github.com/FullStackWithLawrence/aws-openai/settings/secrets/actions), so there's nothing special for you to do.
43+
44+
On the other hand, if you've forked this repo and are working on your own independent project, then you'll need to initialize each of these yourself.
45+
46+
![Github Secrets](https://github.com/FullStackWithLawrence/aws-openai/blob/main/doc/img/github-secrets.png)
47+
48+
## Python Setup
49+
50+
This project includes four distinct Python project, all located in api/terraform/python. They are located here because each of these projects is deployed to AWS Lambda, which in turn is being actively managed by Terraform.
51+
52+
Note that this project leverages Dependabot for managing version numbers of all Python packages that are used in this project, regardless of where and how. Versions should always be up to date at the moment that you clone the repo. It therefore should never be necessary for you to manually bump PyPi package version numbers.
53+
54+
```console
55+
git pull
56+
make init
57+
source venv/bin/activate
58+
```
59+
60+
## Terraform Setup
61+
62+
Please refer to this [Terraform setup guide](https://github.com/FullStackWithLawrence/aws-openai/blob/main/doc/img/TERRAFORM.md) for detailed instructions.
63+
64+
Note that this project leverages Dependabot for managing version numbers of all Terraform modules that are used in this project. Versions should always be up to date at the moment that you clone the repo. It therefore should never be necessary for you to manually bump module version numbers.
65+
66+
```console
67+
git pull
68+
cd terraform
69+
terraform init
70+
terraform plan
71+
terraform apply
72+
```

doc/GOOD_CODING_PRACTICE.md

+50
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Usage of Code Management Best Practices
2+
3+
This microservice fully conforms to [12-Factor methodology](./Twelve_Factor_Methodology.md). Moreover, this repo is used as an instrutional tool for university courses as well as by multiple videos in my [YouTube Channel](https://youtube.com/@FullStackWithLawrence), including various tutorials about good coding practices and good code management. Of note:
4+
5+
## Automations
6+
7+
We leverage automations using Github Actions, third party services, make, bash and anything else that might become freely available to the project in future. As a community-supported code project, automations are important for minimizing the effort required by our committers to keep this code shippable. But it's also an important component of our strategy for maintaining high quality standards. Automations give us the ability to do more work, more consistently, and with less effort.
8+
9+
Of note:
10+
11+
- [Automated Pull Requests](https://github.com/FullStackWithLawrence/aws-openai/pulls?q=is%3Apr+is%3Aclosed): Github Actions are triggered on pull requests to run any of several different kinds of technology-specific unit tests depending on the contents of the commits included in the PR.
12+
- [python-dotenv](https://pypi.org/project/python-dotenv/) for storing sensitive data for local development
13+
- [Pydantic](https://docs.pydantic.dev/latest/) is the most widely used data validation library for Python. We use it for the [Settings](../terraform/python/rekognition_api/conf.py) class in this project. It's an important addition because it enforces strong type and business rule validation checking of all of the configuration parameters for the AWS Rekognition service, and it ensures that nothing ever changes these values at run-time once they've been set. And this in turn is important because erroneous automation code could otherwise lead to some wildly disastrous results. 😳
14+
- [.gitignore](./.gitignore) ensures that no sensitive nor useless data accidentally gets pushed to GitHub.
15+
- [tox.ini](./tox.ini) file for configuring behaviors of Python testing tools
16+
- [GitHub Actions](https://github.com/features/actions) automates unit testing, semantic release rule checking, and dependabot actions.
17+
- [GitHub Secrets](https://github.com/FullStackWithLawrence/aws-openai/settings/secrets/actions) to provide sensitive data to Github Actions workflows
18+
- [GitHub Issues](https://github.com/features/issues)
19+
- [Makefile](./Makefile) automates procedures like init, build, test, release and linting for Python, ReactJS and Terraform.
20+
- [pre-commit](https://pre-commit.com/) automatically enforces a multitude of code quality, coding style and security policies.
21+
- [Dependabot](https://github.com/dependabot) automatically updates the version pins of code library dependencies for Python, ReactJS and Terraform.
22+
- [Unit Tests](https://docs.pytest.org/) are automated and can be invoked
23+
- manually from the command line
24+
- manually from GitHub Actions
25+
- automatically by Dependabot.
26+
- [Mergify](https://mergify.com/) automates processing of bot-created pull requests
27+
- [Semantic Release](https://github.com/semantic-release/semantic-release) automates version releases as well as maintains the change log for the repo.
28+
- [Change Log](http://keepachangelog.com/)
29+
30+
## Linters and Formatters
31+
32+
Linters and formatters are tools used in programming to analyze and improve the quality of code. This project leverages several, including:
33+
34+
### Code Formatting
35+
36+
- [Prettier](https://prettier.io/): an opinionated code formatter that supports many file formats and languages. This project leverages Prettier to standardize formatting of md, css, json, yml, js, jsx and Typescript files.
37+
- [Black](https://github.com/psf/black): an opinionated code formatter for Python which is compatible with [PEP 8](https://peps.python.org/pep-0008/) and the [Python Style Guide](https://www.python.org/doc/essays/styleguide/).
38+
- [isort](https://pycqa.github.io/isort/): a Python utility that sorts imports alphabetically, and automatically, separated into sections and by type.
39+
40+
### Code Analysis
41+
42+
- [ESLint](https://eslint.org/): an open source project that helps you find and fix problems with your JavaScript and JSX code.
43+
- [Flake8](https://flake8.pycqa.org/en/latest/): provides Python syntax checking, naming style enforcement, code style enforcement, and [cyclomatic complexity](https://en.wikipedia.org/wiki/Cyclomatic_complexity) analysis.
44+
- [pylint](https://pypi.org/project/pylint/): a static code analyser for Python. It analyses your code without actually running it. It checks for errors, enforces a coding standard, looks for code smells, and can make suggestions about how the code could be refactored.
45+
- [bandit](https://github.com/PyCQA/bandit): a tool designed to find common security issues in Python code.
46+
47+
### Pre-commit hooks
48+
49+
- [pre-commit Hooks](https://pre-commit.com/hooks.html): scripts that run automatically before each commit is made to a repository, checking your code for embedded passwords, errors, issues, and any of a multitude of configurable policies that you can optionally enforce. They're part of the git hooks system, which allows you to trigger actions at certain points in git's execution. This project uses many Hooks. See [pre-commit-config.yaml](https://github.com/FullStackWithLawrence/aws-openai/blob/main/.pre-commit-config.yaml#L45).
50+
- [codespell](https://github.com/codespell-project/codespell): fixes common misspellings in text files. It's designed primarily for checking misspelled words in source code, but it can be used with other files as well.

doc/README.md

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Technical Overview of this Architecture
2+
3+
- **[Rekognition](https://aws.amazon.com/rekognition/)**: (FOR FUTURE USE) a cloud-based software as a service computer vision platform that was launched in 2016. It is an AWS managed Machine Learning Service with Content moderation, Face compare and search, Face Detection and analysis, Labeling, Custom labels, Text detection, Celebrity recognition, Video segment detection and Streaming Video Events detection features. It is used by a number of United States government agencies, including U.S. Immigration and Customs Enforcement and Orlando, Florida police, as well as private entities.
4+
- **[IAM](https://aws.amazon.com/iam/)**: a web service that helps you securely control access to AWS resources. With IAM, you can centrally manage permissions that control which AWS resources users can access. You use IAM to control who is authenticated (signed in) and authorized (has permissions) to use resources.
5+
- **[S3](https://aws.amazon.com/s3/)**: Amazon Simple Storage Service is a service offered by Amazon Web Services that provides object storage through a web service interface. Amazon S3 uses the same scalable storage infrastructure that Amazon.com uses to run its e-commerce network.
6+
- **[DynamoDB](https://aws.amazon.com/dynamodb/)**: a fully managed proprietary NoSQL database offered by Amazon.com as part of the Amazon Web Services portfolio. DynamoDB offers a fast persistent Key-Value Datastore with built-in support for replication, autoscaling, encryption at rest, and on-demand backup among other features.
7+
- **[Lambda](https://aws.amazon.com/lambda/)**: an event-driven, serverless computing platform provided by Amazon as a part of Amazon Web Services. It is a computing service that runs code in response to events and automatically manages the computing resources required by that code. It was introduced on November 13, 2014.
8+
- **[API Gateway](https://aws.amazon.com/api-gateway/)**: an AWS service for creating, publishing, maintaining, monitoring, and securing REST, HTTP, and WebSocket APIs at any scale.
9+
- **[Certificate Manager](https://aws.amazon.com/certificate-manager/)**: handles the complexity of creating, storing, and renewing public and private SSL/TLS X.509 certificates and keys that protect your AWS websites and applications.
10+
- **[Route53](https://aws.amazon.com/route53/)**: a scalable and highly available Domain Name System service. Released on December 5, 2010.
11+
- **[CloudWatch](https://aws.amazon.com/cloudwatch/)**: CloudWatch enables you to monitor your complete stack (applications, infrastructure, network, and services) and use alarms, logs, and events data to take automated actions and reduce mean time to resolution (MTTR).
12+
13+
## Lambda Functions
14+
15+
These lambdas are designed to work exclusively as back ends for an AWS API Gateway end point resource. PyPi dependencies for these Lambdas are stored in this [AWS Lambda Layer](../api/terraform/python/layer_genai/). Additionally, a locally sourced package named [common](../api/terraform/python/openai_api/common/) is a shared code library used by all Lambdas in this project.
16+
17+
### OpenAI Lambda
18+
19+
Accepts a JSON event with a schema matching the input parameters found in each of the OpenAI API [example application code snippets](https://platform.openai.com/examples/default-grammar/).
20+
21+
### LangChain Lambda
22+
23+
Accepts a simplified JSON schema matching the [LangChain](https://www.langchain.com/) documentation.
24+
25+
## Trouble Shooting and Logging
26+
27+
The terraform scripts will automatically create a collection of CloudWatch Log Groups. Additionally, note the Terraform global variable 'debug_mode' (defaults to 'true') which will increase the verbosity of log entries in the [Lambda functions](./terraform/python/), which are implemented with Python.
28+
29+
I refined the contents and formatting of each log group to suit my own needs while building this solution, and in particular while coding the Python Lambda functions.
30+
31+
![CloudWatch Logs](https://github.com/FullStackWithLawrence/aws-openai/blob/main/doc/img/cloudwatch-1.png "CloudWatch Logs")
32+
![CloudWatch Logs](https://github.com/FullStackWithLawrence/aws-openai/blob/main/doc/img/cloudwatch-2.png "CloudWatch Logs")

doc/SEMANTIC_VERSIONING.md

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Semantic Versioning Guide
2+
3+
See [Conventional Commits](https://www.conventionalcommits.org/)
4+
5+
The [release.yml](.github/workflows/release.yml), [checkPullRequest.yml](.github/workflows/checkPullRequest.yml) and [testRelease.yml](.github/workflows/testRelease.yml) Github Action Workflows in this repo are designed to act on the commit comment rules that follow.
6+
7+
As an open-source maintainer, squash feature branches onto master and write a standardized commit message while doing so. The commit message should be structured as follows:
8+
9+
> `<type>`: This represents the type of change made in the commit. Common types include feat (for a new feature), fix (for a bug fix), chore (for routine tasks like updating dependencies), docs (for documentation changes), style (for code style changes), refactor (for refactoring existing code), test (for adding or updating tests), and perf (for performance improvements).
10+
>
11+
> `[optional scope]`: This is an optional part that provides additional contextual information, like the part of the codebase the commit modifies.
12+
>
13+
> `<description>`: This is a brief description of the changes the commit makes.
14+
>
15+
> `[optional body]`: This is an optional part where you can provide a more detailed explanation of the changes.
16+
>
17+
> `[optional footer(s)]`: This is also optional and is often used to reference issue tracker IDs.
18+
19+
## Commit Message format
20+
21+
- `feat`: A new feature
22+
- `fix`: A bug fix
23+
- `docs`: Documentation only changes
24+
- `style`: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
25+
- `refactor`: A code change that neither fixes a bug nor adds a feature
26+
- `perf`: A code change that improves performance
27+
- `test`: Adding missing or correcting existing tests
28+
- `chore`: Changes to the build process or auxiliary tools and libraries such as documentation generation
29+
30+
Example
31+
32+
```console
33+
git commit -m "docs: add an example of a good commit message"
34+
```

0 commit comments

Comments
 (0)