Skip to content

Commit 636e73c

Browse files
committed
docs(gh-page): update
1 parent 7493f8e commit 636e73c

10 files changed

+280
-170
lines changed

docs/content/en/docs/Developer Guide/configuration.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ These configurations are described in detail in subsequent chapters.
2424
| Configuration | Description | Type | Default | Importance |
2525
|------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|---------------------------------------------------------------------------|------------|
2626
| `fs.listing.class` | Class which is used to list eligible files from the scanned file system. | class | *-* | MEDIUM |
27-
| `fs.listing.filters` | Filters use to list eligible input files | list | *-* | MEDIUM |
27+
| `fs.listing.filters` | A comma-separated list of FileListFilter classes used to list eligible input files. | list | *-* | MEDIUM |
2828
| `fs.listing.interval.ms` | Time interval (in milliseconds) at wish to scan input directory | long | *10000* | HIGH |
2929
| `fs.listing.task.delegation.enabled` | Boolean indicating whether the file listing process should be delegated to tasks. | boolean | *false* | LOW |
3030
| `fs.cleanup.policy.class` | The fully qualified name of the class which is used to cleanup files | class | *-* | HIGH |

docs/content/en/docs/Developer Guide/file-system-listing.md

-169
This file was deleted.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
---
2+
date: 2022-04-13
3+
title: "FileSystem Listing"
4+
linkTitle: "FileSystem Listing"
5+
weight: 30
6+
description: >
7+
Learn how to configure Connect FilePulse for listing files from local or remote storage system.
8+
---
9+
10+
The `FilePulseSourceConnector` periodically lists object files that may be streamed into Kafka using the [FileSystemListing](https://github.com/streamthoughts/kafka-connect-file-pulse/blob/master/connect-file-pulse-api/src/main/java/io/streamthoughts/kafka/connect/filepulse/fs/FileSystemListing.java)
11+
configured in the connector's configuration.
12+
13+
## Supported Filesystems
14+
15+
Currently, Kafka Connect FilePulse supports the following implementations:
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
---
2+
date: 2023-09-12
3+
title: "Alibaba OSS"
4+
linkTitle: "Alibaba OSS"
5+
weight: 60
6+
description: >
7+
Learn how to configure the `AliyunOSSFileSystemListing` to read files on Alibaba OSS.
8+
---
9+
10+
The `AliyunOSSFileSystemListing` class can be used for listing files on Alibaba Cloud Object Storage Service.
11+
12+
## How to use it ?
13+
14+
Use the following property in your Connector's configuration:
15+
16+
`fs.listing.class=io.streamthoughts.kafka.connect.filepulse.fs.AliyunOSSFileSystemListing`
17+
18+
## Configuration
19+
20+
The following table describes the properties that can be used to configure the `AliyunOSSFileSystemListing`:
21+
22+
| Configuration | Description | Type | Default | Importance |
23+
|--------------------------|---------------------------------------------------------------------|----------|---------|------------|
24+
| `oss.access.key.id` | OSS access key id | `string` | - | HIGH |
25+
| `oss.secret.key` | OSS secret key | `int` | - | HIGH |
26+
| `oss.endpoint` | OSS access endpoint | `string` | - | HIGH |
27+
| `oss.bucket.name` | OSS bucket name | `string` | - | HIGH |
28+
| `oss.bucket.prefix` | OSS bucket prefix | `string` | - | HIGH |
29+
| `oss.max.connections` | OSS max connections. | `int` | `1024` | HIGH |
30+
| `oss.socket.timeout` | OSS connection timeout (in milliseconds). | `int` | `10000` | HIGH |
31+
| `oss.connection.timeout` | OSS connection timeout (in milliseconds). | `int` | `50000` | HIGH |
32+
| `oss.max.error.retries` | The maximum number of retry attempts for failed retryable requests. | `int` | `5` | HIGH |

0 commit comments

Comments
 (0)