Skip to content

Commit 56260be

Browse files
arcticflyarcticfly
committed
Big bang
1 parent 978b550 commit 56260be

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

71 files changed

+12493
-20786
lines changed

‎.babelrc

+4-2
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
11
{
22
"presets": [
3-
// "@babel/preset-env"
4-
"@babel/preset-react"
3+
"@babel/preset-env",
4+
"@babel/preset-react",
5+
"@babel/preset-typescript",
6+
"@emotion/babel-preset-css-prop"
57
// "react-app"
68
],
79
"plugins": [

‎.env.example

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
DEBUG_MODE=false

‎.eslintrc

+30-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,34 @@
11
{
2-
"extends": "react-app",
2+
"parser": "@typescript-eslint/parser",
3+
"parserOptions": {
4+
"ecmaVersion": 12,
5+
"sourceType": "module"
6+
},
7+
"plugins": ["@typescript-eslint", "prettier", "unused-imports"],
8+
"extends": [
9+
"eslint:recommended",
10+
"plugin:@typescript-eslint/recommended",
11+
"prettier"
12+
],
13+
"rules": {
14+
"prettier/prettier": "error",
15+
"@typescript-eslint/no-unused-vars": "off",
16+
"unused-imports/no-unused-imports": "error",
17+
"unused-imports/no-unused-vars": [
18+
"warn",
19+
{
20+
"vars": "all",
21+
"varsIgnorePattern": "^_",
22+
"args": "after-used",
23+
"argsIgnorePattern": "^_"
24+
}
25+
]
26+
},
27+
"env": {
28+
"browser": true,
29+
"es2021": true,
30+
"node": true
31+
},
332
"globals": {
433
"chrome": "readonly"
534
}

‎.github/workflows/main.yml

+48
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
name: Build and Release Chrome Extension
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
tags:
8+
- 'v*.*.*'
9+
10+
jobs:
11+
build-and-release:
12+
runs-on: ubuntu-latest
13+
permissions:
14+
contents: write
15+
16+
steps:
17+
- name: Checkout code
18+
uses: actions/checkout@v3
19+
20+
- name: Set up Node.js
21+
uses: actions/setup-node@v2
22+
with:
23+
node-version: 16
24+
25+
- name: Install dependencies and build extension
26+
run: |
27+
yarn install
28+
NODE_ENV=production yarn build
29+
30+
- name: Package extension
31+
id: packExtensionDir
32+
uses: cardinalby/webext-buildtools-pack-extension-dir-action@v1
33+
with:
34+
extensionDir: 'build' # change this to your extension build folder
35+
zipFilePath: 'build/extension.zip'
36+
37+
- name: Sign Chrome extension
38+
uses: cardinalby/webext-buildtools-chrome-crx-action@v2
39+
with:
40+
zipFilePath: 'build/extension.zip'
41+
crxFilePath: 'build/extension.crx'
42+
privateKey: ${{ secrets.CHROME_CRX_PRIVATE_KEY }}
43+
44+
- name: Release
45+
uses: softprops/action-gh-release@v1
46+
if: startsWith(github.ref, 'refs/tags/')
47+
with:
48+
files: 'build/extension.crx'

‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111

1212
# misc
1313
.DS_Store
14+
.env
1415
.env.local
1516
.env.development.local
1617
.env.test.local

‎.tool-versions

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
nodejs 19.7.0

‎.vscode/settings.json

+5-2
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
11
{
2-
"editor.formatOnSave": true
3-
}
2+
"eslint.format.enable": true,
3+
"editor.codeActionsOnSave": {
4+
"source.fixAll.eslint": true
5+
}
6+
}

‎LICENSE

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
The MIT License (MIT)
22

3-
Copyright (c) 2019 Michael Xieyang Liu
3+
Copyright (c) 2023 Taxy AI
44

55
Permission is hereby granted, free of charge, to any person obtaining a copy
66
of this software and associated documentation files (the "Software"), to deal
@@ -18,4 +18,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
1818
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
1919
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
2020
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21-
SOFTWARE.
21+
SOFTWARE.

‎README.md

+73-116
Original file line numberDiff line numberDiff line change
@@ -1,151 +1,108 @@
11
<img src="src/assets/img/icon-128.png" width="64"/>
22

3-
# Chrome Extension Boilerplate with React 17 and Webpack 5
3+
# Taxy AI: Full Browser Automation
4+
[Waitlist](https://docs.google.com/forms/d/e/1FAIpQLScAFKI1fZ1cXhBmSp2HM93Jvuc8Jvrxh5iSbkKhtwKN-OHoTQ/viewform) | [Discord](https://discord.gg/DXaErbBc)
45

5-
[![npm](https://img.shields.io/npm/v/chrome-extension-boilerplate-react)](https://www.npmjs.com/package/chrome-extension-boilerplate-react)
6-
[![npm-download](https://img.shields.io/npm/dw/chrome-extension-boilerplate-react)](https://www.npmjs.com/package/chrome-extension-boilerplate-react)
7-
[![npm](https://img.shields.io/npm/dm/chrome-extension-boilerplate-react)](https://www.npmjs.com/package/chrome-extension-boilerplate-react)
6+
Taxy uses GPT-4 to control your browser and perform repetitive actions on your behalf. Currently it allows you to define ad-hoc instructions. In the future it will also support saved and scheduled workflows.
87

9-
[![dependencies Status](https://david-dm.org/lxieyang/chrome-extension-boilerplate-react/status.svg)](https://david-dm.org/lxieyang/chrome-extension-boilerplate-react)
10-
[![devDependencies Status](https://david-dm.org/lxieyang/chrome-extension-boilerplate-react/dev-status.svg)](https://david-dm.org/lxieyang/chrome-extension-boilerplate-react?type=dev)
8+
Taxy's current status is **research preview**. Many workflows fail or confuse the agent. If you'd like to hack on Taxy to make it better or test it on your own workflows, follow the instructions below to run it locally. If you'd like to know once it's available for wider usage, you can sign up for our [waitlist](https://docs.google.com/forms/d/e/1FAIpQLScAFKI1fZ1cXhBmSp2HM93Jvuc8Jvrxh5iSbkKhtwKN-OHoTQ/viewform).
119

12-
## Announcements
10+
Taxy is fully open-source, and we don't send any page contents or instructions to our servers.
1311

14-
- **_This boilerplate adopts [Manifest V3](https://developer.chrome.com/docs/extensions/mv3/intro/mv3-overview/)!_** For V2 users, please check out the [manifest-v2](https://github.com/lxieyang/chrome-extension-boilerplate-react/tree/manifest-v2) branch, or use version [3.x](https://www.npmjs.com/package/chrome-extension-boilerplate-react/v/3.3.0).
15-
- Check out the [Manifest V3 Migration Guide](https://developer.chrome.com/docs/extensions/mv3/intro/mv3-migration/).
16-
- Recently added [devtools](https://developer.chrome.com/docs/extensions/mv3/devtools/) Support! Thanks [GeekaholicLin](https://github.com/lxieyang/chrome-extension-boilerplate-react/issues/17)!
17-
- Recently updated from **[React](https://reactjs.org)** ~~16~~ to **17**!
18-
- Recently updated from **[Webpack Dev Server](https://webpack.js.org/configuration/dev-server/)** ~~3.x~~ to **4.x** and **[Webpack](https://webpack.js.org/)** ~~4~~ to **5**!
19-
- Recently added [TypeScript](https://www.typescriptlang.org/) Support!
12+
Here's Taxy using Google Calendar with the prompt "Schedule standup tomorrow at 10am. Invite [email protected]"
2013

21-
## Features
14+
![calendar](https://user-images.githubusercontent.com/176426/228092695-1bc11ea9-bfb7-470d-bbc6-0026e93c23c3.gif)
2215

23-
This is a basic Chrome Extensions boilerplate to help you write modular and modern Javascript code, load CSS easily and [automatic reload the browser on code changes](https://webpack.github.io/docs/webpack-dev-server.html#automatic-refresh).
2416

25-
This boilerplate is updated with:
17+
## Table of Contents
2618

27-
- [Chrome Extension Manifest V3](https://developer.chrome.com/docs/extensions/mv3/intro/mv3-overview/)
28-
- [React 17](https://reactjs.org)
29-
- [Webpack 5](https://webpack.js.org/)
30-
- [Webpack Dev Server 4](https://webpack.js.org/configuration/dev-server/)
31-
- [React Hot Loader](https://github.com/gaearon/react-hot-loader)
32-
- [eslint-config-react-app](https://www.npmjs.com/package/eslint-config-react-app)
33-
- [Prettier](https://prettier.io/)
34-
- [TypeScript](https://www.typescriptlang.org/)
35-
36-
This boilerplate is heavily inspired by and adapted from [https://github.com/samuelsimoes/chrome-extension-webpack-boilerplate](https://github.com/samuelsimoes/chrome-extension-webpack-boilerplate), with additional support for React 17 features, Webpack 5, and Webpack Dev Server 4.
37-
38-
Please open up an issue to nudge me to keep the npm packages up-to-date. FYI, it takes time to make different packages with different versions work together nicely.
19+
- [Taxy AI: Full Browser Automation](#taxy-ai-full-browser-automation)
20+
- [Table of Contents](#table-of-contents)
21+
- [Installing and Running](#installing-and-running)
22+
- [Installing the extension](#installing-the-extension)
23+
- [Running in your browser](#running-in-your-browser)
24+
- [How it Works - The Action Cycle](#how-it-works---the-action-cycle)
25+
- [Simple Demos](#simple-demos)
26+
- [Protecting the main branch in GitHub](#protecting-the-main-branch-in-github)
27+
- [Searching for and playing the movie Oblivion in Netflix](#searching-for-and-playing-the-movie-oblivion-in-netflix)
28+
- [Creating a calendar event](#creating-a-calendar-event)
29+
- [Writing an essay in the OpenAI playground](#writing-an-essay-in-the-openai-playground)
30+
- [Add your own!](#add-your-own)
31+
- [Tech Stack](#tech-stack)
32+
- [Resources](#resources)
3933

4034
## Installing and Running
4135

42-
### Procedures:
43-
44-
1. Check if your [Node.js](https://nodejs.org/) version is >= **14**.
45-
2. Clone this repository.
46-
3. Change the package's `name`, `description`, and `repository` fields in `package.json`.
47-
4. Change the name of your extension on `src/manifest.json`.
48-
5. Run `npm install` to install the dependencies.
49-
6. Run `npm start`
50-
7. Load your extension on Chrome following:
51-
1. Access `chrome://extensions/`
52-
2. Check `Developer mode`
53-
3. Click on `Load unpacked extension`
54-
4. Select the `build` folder.
55-
8. Happy hacking.
56-
57-
## Structure
58-
59-
All your extension's code must be placed in the `src` folder.
60-
61-
The boilerplate is already prepared to have a popup, an options page, a background page, and a new tab page (which replaces the new tab page of your browser). But feel free to customize these.
62-
63-
## TypeScript
64-
65-
This boilerplate now supports TypeScript! The `Options` Page is implemented using TypeScript. Please refer to `src/pages/Options/` for example usages.
36+
Currently this extension is only available through this GitHub repo. We'll release it on the Chrome Web Store after adding features to increase its usability for a non-technical audience. To build and install the extension locally on your machine, follow the instructions below.
6637

67-
## Webpack auto-reload and HRM
38+
### Installing the extension
6839

69-
To make your workflow much more efficient this boilerplate uses the [webpack server](https://webpack.github.io/docs/webpack-dev-server.html) to development (started with `npm start`) with auto reload feature that reloads the browser automatically every time that you save some file in your editor.
70-
71-
You can run the dev mode on other port if you want. Just specify the env var `port` like this:
72-
73-
```
74-
$ PORT=6002 npm run start
75-
```
76-
77-
## Content Scripts
78-
79-
Although this boilerplate uses the webpack dev server, it's also prepared to write all your bundles files on the disk at every code change, so you can point, on your extension manifest, to your bundles that you want to use as [content scripts](https://developer.chrome.com/extensions/content_scripts), but you need to exclude these entry points from hot reloading [(why?)](https://github.com/samuelsimoes/chrome-extension-webpack-boilerplate/issues/4#issuecomment-261788690). To do so you need to expose which entry points are content scripts on the `webpack.config.js` using the `chromeExtensionBoilerplate -> notHotReload` config. Look the example below.
80-
81-
Let's say that you want use the `myContentScript` entry point as content script, so on your `webpack.config.js` you will configure the entry point and exclude it from hot reloading, like this:
82-
83-
```js
84-
{
85-
…
86-
entry: {
87-
myContentScript: "./src/js/myContentScript.js"
88-
},
89-
chromeExtensionBoilerplate: {
90-
notHotReload: ["myContentScript"]
91-
}
92-
…
93-
}
94-
```
40+
1. Ensure you have [Node.js](https://nodejs.org/) >= **16**.
41+
2. Clone this repository
42+
3. Run `yarn` to install the dependencies
43+
4. Run `yarn start` to build the package
44+
5. Load your extension on Chrome by doing the following:
45+
1. Navigate to `chrome://extensions/`
46+
2. Toggle `Developer mode`
47+
3. Click on `Load unpacked extension`
48+
4. Select the `build` folder that `yarn start` generated
9549

96-
and on your `src/manifest.json`:
50+
### Running in your browser
9751

98-
```json
99-
{
100-
"content_scripts": [
101-
{
102-
"matches": ["https://www.google.com/*"],
103-
"js": ["myContentScript.bundle.js"]
104-
}
105-
]
106-
}
107-
```
52+
1. Once installed, the browser plugin will be available in two forms:
53+
1. As a Popup. Activate by pressing `cmd+shift+y` on mac or `ctrl+shift+y` on windows/linux, or by clicking the extension logo in your browser.
54+
2. As a devtools panel. Activate by first opening the browser's developer tools, then navigating to the `Taxy AI` panel.
55+
2. The next thing you need to do is create or access an existing [OpenAI API Key](https://platform.openai.com/account/api-keys) and paste it in the provided box. This key will be stored securely in your browser, and will not be uploaded to a third party.
56+
3. Finally, navigate to a webpage you want Taxy to act upon (for instance the [OpenAI playground](https://platform.openai.com/playground)) and start experimenting!
10857

109-
## Intelligent Code Completion
58+
## How it Works - The Action Cycle
11059

111-
Thanks to [@hudidit](https://github.com/lxieyang/chrome-extension-boilerplate-react/issues/4)'s kind suggestions, this boilerplate supports chrome-specific intelligent code completion using [@types/chrome](https://www.npmjs.com/package/@types/chrome).
60+
1. Taxy runs a content script on the webpage to pull the entire DOM. It simplifies the html it receives to only include interactive or semantically important elements, like buttons or text. It assigns an id to each interactive element. It then "templatizes" the DOM to reduce the token count even further.
61+
2. Taxy sends the simplified DOM, along with the user's instructions, to a selected LLM (currently GPT-3.5 and GPT-4 are supported). Taxy informs the LLM of two methods to interact with the webpage:
62+
1. `click(id)` - click on the interactive element associated with that id
63+
2. `setValue(id, text)` - focus on a text input, clear its existing text, and type the specified text into that input
64+
3. When Taxy gets a completion from the LLM, it parses the response for an action. The action cycle will end at this stage if any of the following conditions are met:
65+
1. The LLM believes the task is complete. Instead of an action, the LLM can return an indication that it believes the user's task is complete based on the state of the DOM and the action history up to this point.
66+
2. The user stopped the task's execution. The user can stop the LLM's execution at any time, without waiting for it to complete.
67+
3. There was an error. Taxy's safety-first architecture causes it to automatically halt execution in the event of an unexpected response.
68+
4. Taxy executes the action using the [chrome.debugger API](https://developer.chrome.com/docs/extensions/reference/debugger/).
69+
5. The action is added to the action history and Taxy cycles back to step 1 and parses the updated DOM. All prior actions are sent to the LLM as part of the prompt used to determine the next action. Taxy can currently complete a maximum of 50 actions for a single task, though in practice most tasks require fewer than 10 actions.
11270

113-
## Packing
71+
## Simple Demos
11472

115-
After the development of your extension run the command
73+
### Protecting the main branch in GitHub
11674

117-
```
118-
$ NODE_ENV=production npm run build
119-
```
75+
![main-branch](https://user-images.githubusercontent.com/41524992/228385404-175bf633-de1d-43eb-862c-2cfd5a7a674a.gif)
12076

121-
Now, the content of `build` folder will be the extension ready to be submitted to the Chrome Web Store. Just take a look at the [official guide](https://developer.chrome.com/webstore/publish) to more infos about publishing.
77+
### Searching for and playing the movie Oblivion in Netflix
12278

123-
## Secrets
79+
![Oblivion](https://user-images.githubusercontent.com/41524992/228063949-c26a4b54-92ae-4e22-8177-7e78c0d9a29b.gif)
12480

125-
If you are developing an extension that talks with some API you probably are using different keys for testing and production. Is a good practice you not commit your secret keys and expose to anyone that have access to the repository.
81+
### Creating a calendar event
12682

127-
To this task this boilerplate import the file `./secrets.<THE-NODE_ENV>.js` on your modules through the module named as `secrets`, so you can do things like this:
83+
![Calendar](https://user-images.githubusercontent.com/41524992/228064011-0ca3a39d-cebb-4a55-9e2b-6aa3ae5b3f43.gif)
12884

129-
_./secrets.development.js_
85+
### Writing an essay in the OpenAI playground
13086

131-
```js
132-
export default { key: '123' };
133-
```
87+
![Playground](https://user-images.githubusercontent.com/41524992/228064056-84eab4e4-b5b5-4e79-b1e3-be92f14d2607.gif)
13488

135-
_./src/popup.js_
89+
### Add your own!
13690

137-
```js
138-
import secrets from 'secrets';
139-
ApiCall({ key: secrets.key });
140-
```
91+
If you have an interesting demo you'd like to share, submit a PR to add your own!
14192

142-
:point_right: The files with name `secrets.*.js` already are ignored on the repository.
93+
## Tech Stack
14394

144-
## Resources:
95+
Technology currently used by this extension:
14596

146-
- [Webpack documentation](https://webpack.js.org/concepts/)
147-
- [Chrome Extension documentation](https://developer.chrome.com/extensions/getstarted)
97+
- [Chrome Extension Manifest V3](https://developer.chrome.com/docs/extensions/mv3/intro/mv3-overview/)
98+
- [React 17](https://reactjs.org)
99+
- [Webpack 5](https://webpack.js.org/)
100+
- [Webpack Dev Server 4](https://webpack.js.org/configuration/dev-server/)
101+
- [React Hot Loader](https://github.com/gaearon/react-hot-loader)
102+
- [eslint-config-react-app](https://www.npmjs.com/package/eslint-config-react-app)
103+
- [Prettier](https://prettier.io/)
104+
- [TypeScript](https://www.typescriptlang.org/)
148105

149-
---
106+
## Resources
150107

151-
Michael Xieyang Liu | [Website](https://lxieyang.github.io)
108+
- [Getting Started with Chrome Extensions](https://developer.chrome.com/extensions/getstarted)

‎demos/recording-to-gif.sh

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
#!/bin/bash
2+
3+
# Check if user provided a file name
4+
if [ -z "$1" ]; then
5+
echo "Please provide a video file name"
6+
exit 1
7+
fi
8+
9+
# Set default scale-down factor (proportional to video size)
10+
scale=2
11+
12+
# Check if user provided a scale-down parameter
13+
if [ ! -z "$2" ]; then
14+
scale=$2
15+
fi
16+
17+
# By default, speed up the video by 4x
18+
shorten_factor=0.25
19+
20+
# Check if user provided a speed-up parameter
21+
if [ ! -z "$3" ]; then
22+
shorten_factor=$3
23+
fi
24+
25+
# Get input file name without extension
26+
input_file="${1%.*}"
27+
28+
# Set output file name with .gif extension
29+
output_file="$input_file.gif"
30+
31+
# Use FFmpeg to make the GIF
32+
ffmpeg -i "$1" -vf "setpts=${shorten_factor}*PTS,fps=15,scale=iw/${scale}:ih/${scale}:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" -loop 0 "$output_file"
33+
34+
echo "GIF conversion complete. Output file: $output_file"

0 commit comments

Comments
 (0)