Skip to content

[Edit] Python (Pytorch): .cat() #6485

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Apr 15, 2025
45 changes: 39 additions & 6 deletions content/pytorch/concepts/tensors/terms/cat/cat.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,33 +2,38 @@
Title: '.cat()'
Description: 'Concatenates two or more tensors in the same dimension.'
Subjects:
- 'AI'
- 'Computer Science'
- 'Data Science'
Tags:
- 'AI'
- 'Deep Learning'
- 'Functions'
- 'Machine Learning'
- 'PyTorch'
CatalogContent:
- 'intro-to-py-torch-and-neural-networks'
- 'py-torch-for-classification'
---

The **`.cat()`** function in PyTorch concatenates two or more tensors along a specified dimension. The tensors must have the same shape in all dimensions except for the dimension along which they are concatenated.
The **`.cat()`** function in PyTorch concatenates two or more [tensors](https://www.codecademy.com/resources/docs/pytorch/tensors) along a specified dimension. The tensors must have the same shape in all dimensions except for the dimension along which they are concatenated.

## Syntax

```pseudo
torch.cat(tensors, dim=0, out=None)
```

## Parameters
**Parameters:**

- `tensors`: A sequence (like a list or tuple) of tensors to be concatenated. All tensors must have the same shape in all dimensions except for the specified dimension.
- `dim`: An integer specifying the dimension along which the tensors will be concatenated. The default value is `0`, which means concatenation will occur along the first dimension.
- `out`: A pre-allocated tensor with the correct shape to store the result of the concatenation. If not provided, a new tensor will be allocated.

## Example 1
**Return value:**

A new tensor resulting from concatenating the input tensors along the specified dimension.

## Example 1: Concatenating tensors along the first dimension
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line -"The output shows the two tensors and the concatenated tensor along the first dimension:" is repetitive in all three examples. Consider rewriting it. We are saying the same thing in the example heading, then description of example and output description.


The example below showcases concatenating tensors along the first dimension using the `.cat()` function:

Expand Down Expand Up @@ -70,7 +75,7 @@ tensor([[ 1, 2, 3],
[10, 11, 12]])
```

## Example 2
## Example 2: Concatenating tensors along the second dimension

The example below showcases concatenating tensors along the second dimension using the `.cat()` function:

Expand Down Expand Up @@ -110,7 +115,7 @@ tensor([[ 1, 2, 3, 7, 8, 9],
[ 4, 5, 6, 10, 11, 12]])
```

## Example 3
## Example 3: Concatenating tensors along the third dimension

The example below showcases concatenating tensors along the third dimension using the `.cat()` function:

Expand Down Expand Up @@ -158,3 +163,31 @@ tensor([[[ 1, 2, 3, 13, 14, 15],
[[ 7, 8, 9, 19, 20, 21],
[10, 11, 12, 22, 23, 24]]])
```

## Best Practices for using `.cat()` in PyTorch

1. **Match Shapes Across All Dimensions Except the Concatenation Axis:** Ensure all tensors have the same size in every dimension except the one you're concatenating along. Mismatched dimensions will cause errors.

2. **Use a List or Tuple of Tensors:** Group tensors into a list or tuple when passing them to `.cat()` for cleaner, more readable code—especially when combining multiple tensors.

3. **Avoid Unnecessary In-Place Operations:** Although torch.cat() supports an out parameter, it's usually best to let it return a new tensor unless memory usage is a concern.

## FAQs

<details>
<summary>1. What is the difference between torch.stack() and `.cat()`?</summary>
- `.cat()` combines tensors along an existing dimension.
- `.stack()` adds a new dimension and stacks tensors along that new axis.
Use stack when you want to create a new level of nesting; use cat to extend an existing one.
</details>

<details>
<summary>2. Can I concatenate tensors of different data types or devices?</summary>
<p>No. All tensors must have the same data type and must be on the same device (e.g., all on CPU or all on GPU). Mismatches will raise an error.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can add backlink to CPU and GPU

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have one for GPU, added 1cpu blog link (It has better content than docs)

</p>
</details>

<details>
<summary>3. Can I concatenate along any dimension?</summary>
<p>Yes, as long as all other dimensions match. You can concatenate along any valid axis that exists in the input tensors.</p>
</details>