Refine test suite

[ehmatthes]

The initial test suite has been developed, and is starting to become useful in ongoing development work. There are a few smaller tasks that would help improve the test suite, so it doesn't take any more maintenance attention than it needs.

Keep in mind that principles like DRY don't apply the same to testing as they do to general software development. When a test fails, you really want that test to be as easy to understand as possible.

Let's aim to make some refinements, and then close this so that most of the focus is on ongoing development.

• List any refinements that we'd like to make.
• Implement these refinements.
• Make sure tests pass on all major OSes: Windows, macOS, and Linux.

[ehmatthes]

Initial refinements

• Consider using opencv in compare_images()? It's already a dependency of the project; it may provide a better measure of the correctness of generated images.
• Consider writing a helper function that reduces the repetition in test_core_commands.py. It's probably better to have many test functions than a few with parametrization, but a helper function that takes in raw_cmd might be worth looking at. See Use parametrization to simplify tests. #100.
• Add a test for a named font, because most tests will use a path to a font.
• Update docs to reflect current state of testing.
  • Update cross-platform discussion.
• Update Contributing docs, or project structure so that a new contributor can follow instructions on any OS and have a passing test suite. See notes about Windows here.
• Add a pytest flag or plugin that rewrites all reference files. See here for current semi-automated approach.
• Add a pytest flag or plugin that opens a terminal at the sample repo, and shows the command to run a git-sim command in that repo.
• Consider capturing some of the unfinished tasks from Design and create a test suite #55 in a non-issue doc, ie continuous integration, testing CLI output.
• Write any additional tests that are obviously helpful, such as light mode, one test for jpg.
• Add a --test-current-release flag that will install git-sim from PyPI instead of using the local editable install. This can be nice to run immediately after making a release, and you can test different package managers as well.

[micwoj92]

Could you consider running tests without using venv for this refinement?