Add tutorial about inductor caching

[oulgen]

Stack from ghstack (oldest at bottom):

• -> Add tutorial about inductor caching #2951

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/2951

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Failure with setup-ssh on Amazon Linux 2023 runners

✅ No Failures

As of commit e6b6dd2 with merge base f2b8a1b ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[svekars]

A quick editorial pass - let me know if you have any questions.

[svekars]

Suggested change

	This recipes demonstrates how you to configure various parts of the caching in ``torch.compile``.
	This recipe demonstrates how you can configure various parts of the caching in ``torch.compile``.

[svekars]

Suggested change

Most of these caches are in-memory, only used within the same process, and are transparent to the user. An exception is the FX graph cache that stores compiled FX graphs. This cache allows Inductor to avoid recompilation across process boundaries when it encounters the same graph with the same Tensor input shapes (and the same configuration, etc.). The default implementation stores compiled artifacts in the system temp directory. An optional feature also supports sharing those artifacts within a cluster by storing them in Redis.

Most of these caches are in-memory, only used within the same process, and are transparent to the user. An exception is the FX graph cache that stores compiled FX graphs. This cache allows Inductor to avoid recompilation across process boundaries when it encounters the same graph with the same Tensor input shapes (and the same configuration). The default implementation stores compiled artifacts in the system temp directory. An optional feature also supports sharing those artifacts within a cluster by storing them in a Redis database.

[svekars]

Suggested change

	There are a few settings relevant to caching and to FX graph caching in particular. The settings are accessible via environment variables, or can be hard-coded in Inductor’s config file.
	There are a few settings relevant to caching and to FX graph caching in particular.
	The settings are accessible via environment variables listed below or can be hard-coded in the Inductor’s config file.

[masnesral]

But I'd do: "the Inductor's config file" -> "Inductor's config file"

[svekars]

Suggested change

	Note that if ``TRITON_CACHE_DIR`` is not set in the environment, Inductor sets the Triton cache directory to this same temp location (under the Triton subdirectory).
	Note that if ``TRITON_CACHE_DIR`` is not set in the environment, Inductor sets the Triton cache directory to this same temporary location, under the Triton subdirectory.

[svekars]

Suggested change

	This setting specifies the location of all on-disk caches. By default, the location is in the system temp directory under ``torchinductor_<username>``, e.g., ``/tmp/torchinductor_myusername``.
	This setting specifies the location of all on-disk caches. By default, the location is in the system temporary directory under ``torchinductor_<username>``, for example, ``/tmp/torchinductor_myusername``.

[svekars]

Suggested change

	This setting enables a remote cache for Inductor’s autotuner. As with the remote FX graph cache, the current implementation uses Redis. ``1`` enables, and any other value disables. The same host / port environment variables listed above apply to this cache.
	This setting enables a remote cache for Inductor’s autotuner. As with the remote FX graph cache, the current implementation uses Redis. ``1`` enables cache, and any other value disables. The same host / port environment variables listed above apply to this cache.

[masnesral]

But I do: enables cache -> enables caching

[svekars]

Suggested change

	Set this value to ``1`` to disable all Inductor caching. This setting is useful to, e.g., experiment with cold-start compile time, or to force recompilation for debugging purposes.
	Set this value to ``1`` to disable all Inductor caching. This setting is useful for tasks like experimenting with cold-start compile times or forcing recompilation for debugging purposes..

[svekars]

Can we also add a Conclusion section, something like:

In this recipe, we have learned that PyTorch Inductor's caching mechanisms significantly
reduce compilation latency by utilizing both local and remote caches, which operate seamlessly in the background without requiring user intervention. Additionally, we explored the various settings and
environment variables that allow users to configure and optimize these caching features according
to their specific needs.

[masnesral]

Inductor's config file

[oulgen]

Apparently I should not be using ghstack so closing this and creating a new PR: #2952

[masnesral]

enables caching

[masnesral]

disables it