Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added .github/images/site-ci-overview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
214 changes: 160 additions & 54 deletions .github/workflows/hugo.yml

Large diffs are not rendered by default.

25 changes: 2 additions & 23 deletions assets/js/kapa-adapter.js
Original file line number Diff line number Diff line change
Expand Up @@ -141,8 +141,6 @@
// Storage may be disabled; consent still works for this page only.
}
var consented = choice === 'granted';
var grantPersisted = consented;
var dismissed = choice === 'dismissed';
windowObject.addEventListener('storage', function (event) {
if (event.key !== consentKey && event.key !== null) return;
if ((consented && event.newValue !== 'granted') ||
Expand All @@ -155,14 +153,14 @@
var pending = null;
var consent = documentObject.querySelector('[data-hg-ai-consent]');
var launcher = documentObject.querySelector('.hg-ask-ai-launcher');
if (consent) consent.hidden = consented || dismissed;
if (consent) consent.hidden = true;
function renderLauncher(hidden) {
if (!launcher) return;
launcher.hidden = hidden;
if (consented) launcher.setAttribute('aria-haspopup', 'dialog');
else launcher.removeAttribute('aria-haspopup');
}
renderLauncher(!consented && !dismissed);
renderLauncher(false);

function dismissConsent() {
if (consent) consent.hidden = true;
Expand All @@ -175,23 +173,6 @@
var activeQueue = null;
var operation = null;
var status = documentObject.querySelector('[data-hg-ai-status]');
var revoke = documentObject.querySelector('[data-hg-ai-revoke]');
if (revoke) {
revoke.hidden = !consented;
revoke.addEventListener('click', function () {
if (!consented) return;
if (grantPersisted) {
try {
windowObject.localStorage.removeItem(consentKey);
} catch (_) {
renderState(state, config.labels.revokeError);
return;
}
}
// Navigation terminates the vendor's loaded state and pending callbacks.
windowObject.location.reload();
});
}

function renderState(next, message) {
state = next;
Expand Down Expand Up @@ -412,11 +393,9 @@
consented = true;
try {
windowObject.localStorage.setItem(consentKey, 'granted');
grantPersisted = true;
} catch (_) {
// Never bypass initial consent when persistence is unavailable.
}
if (revoke) revoke.hidden = false;
dismissConsent();
load(request.query, request.submit);
});
Expand Down
29 changes: 25 additions & 4 deletions assets/scss/_styles_project.scss
Original file line number Diff line number Diff line change
Expand Up @@ -865,12 +865,33 @@
}


// The first-use notice stays beside the AI entry without blocking the page.
.hg-ai-revoke {
font-size: inherit;
&[hidden] { display: none; }
// Let documentation results scroll without covering them with the AI action.
.td-shell-search__list:has(> [aria-labelledby="td-shell-search-group-actions"]) {
display: flex;
flex-direction: column;
overflow: hidden;

> .td-shell-search__group {
min-height: 0;
overflow-y: auto;
overscroll-behavior: contain;
}
}

.td-shell-search__group[aria-labelledby="td-shell-search-group-actions"] {
flex-shrink: 0;
background: var(--bs-body-bg);
border-top: 1px solid var(--bs-border-color);
box-shadow: 0 -.25rem .75rem rgba(0, 0, 0, .06);

.td-shell-search__item {
color: var(--bs-primary);
background: rgba(var(--bs-primary-rgb), .08);
}
}

// The first-use notice opens beside the AI entry after explicit activation.

.hg-ai-consent {
position: fixed;
z-index: 1040;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@ weight: 2

如果你需要把数据从一张 HugeGraph 图复制到另一张图,使用 `graph2graph`:HugeGraph Source 从源图(A 图)读取顶点和边,数据经过可选的 Transform 后,由 HugeGraph Sink 写入目标图(B 图)。数据方向是 `A 图 → HugeGraph Source →(可选 Transform)→ HugeGraph Sink → B 图`。如果要把图数据导出到文件、JDBC、Kafka 等其他系统,则使用 `graph2any`,由下游 Sink 接收 Source 读取的数据。本文介绍这两类任务。

> **版本要求:本文面向 SeaTunnel [3.0+](https://github.com/apache/seatunnel/tree/3.0.0-release)**
> **版本要求:本文面向 SeaTunnel [3.0+](https://seatunnel.apache.org/download/)**

SeaTunnel 3.0+ 提供 HugeGraph Source,支持图模型自动发现和多标签读取。
并行扫描的后端要求和配置限制见[官方 Source 文档](https://seatunnel.apache.org/docs/connectors/source/HugeGraph/)。

开始前请先完成[导入页中的通用环境准备和配置](/cn/docs/quickstart/toolchain/import/hugegraph-seatunnel-connector/#2-准备环境),其中包含 JDK、HOCON、插件安装和图模型说明。

Expand Down Expand Up @@ -138,7 +141,7 @@ sink {

> **为什么保留 ID?** HugeGraph 的 `PRIMARY_KEY` ID 包含顶点标签的内部 ID,两张图可能不同。例如源图顶点是 `1:marko`,目标图重新按主键生成的可能是 `2:marko`。如果重新生成顶点 ID 后仍复用源图的边端点,边就会连错。本例将原 ID 保存为字符串,因此会改变目标图的 ID 策略

若要一次读取全部标签,省略 Source 的 `label` 后会读取 `label_type`(默认 `VERTEX`)下的全部 label,每个 label 输出一张表。这时需用 `sourceTable` 将各 Sink 映射绑定到对应表,例如 `sourceTable = "default.person"`;具体值以 Writer 日志中的完整表名为准。不能直接套用本节的单标签配置。其他限制见 [HugeGraph Source 文档](https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/connectors/source/HugeGraph.md)。
若要一次读取全部标签,省略 Source 的 `label` 后会读取 `label_type`(默认 `VERTEX`)下的全部 label,每个 label 输出一张表。这时需用 `sourceTable` 将各 Sink 映射绑定到对应表,例如 `sourceTable = "default.person"`;具体值以 Writer 日志中的完整表名为准。不能直接套用本节的单标签配置。其他限制见 [HugeGraph Source 文档](https://seatunnel.apache.org/docs/connectors/source/HugeGraph/)。

## 2 导出到其他系统(graph2any)

Expand Down Expand Up @@ -188,8 +191,8 @@ sink {

**连接器**

<p><sup>[1]</sup> <a href="https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/connectors/source/HugeGraph.md">HugeGraph Source</a><br>
<sup>[2]</sup> <a href="https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/connectors/sink/LocalFile.md">LocalFile Sink</a></p>
<p><sup>[1]</sup> <a href="https://seatunnel.apache.org/docs/connectors/source/HugeGraph/">HugeGraph Source</a><br>
<sup>[2]</sup> <a href="https://seatunnel.apache.org/docs/connectors/sink/LocalFile/">LocalFile Sink</a></p>

**关联文档**

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ aliases:

SeaTunnel 可以把数据库、Kafka 等数据源接入 HugeGraph。连接器分为两部分:**Source 负责读取,Sink 负责写入**<sup>[1][2]</sup>,中间可以接 SeaTunnel 的数据转换组件。需要从 HugeGraph 导出或迁移数据时,请查看[SeaTunnel Source 导出与迁移文档](/cn/docs/quickstart/toolchain/export-migration/hugegraph-seatunnel-source/)。

> **版本要求:本文面向 SeaTunnel [3.0+](https://github.com/apache/seatunnel/tree/3.0.0-release)。** 所有示例使用 `mappings`
> **版本要求:本文面向 SeaTunnel [3.0+](https://seatunnel.apache.org/download/)。** 所有示例使用 SeaTunnel 3.0+ 提供的 `mappings` 配置。

[![Loader 与 SeaTunnel 的工作流对比:直接导入图数据与复用 Source、Transform、Sink 管道](/cn/docs/images/seatunnel/seatunnel-vs-loader-zh.png)](/cn/docs/images/seatunnel/seatunnel-vs-loader-zh.png)

Expand Down Expand Up @@ -38,25 +38,45 @@ SeaTunnel 覆盖 Loader 的图导入和 Tools 的导出、迁移场景,可以

> **已有 Spark/Flink 每日任务**
>
> HugeGraph Source 和 Sink 在 SeaTunnel 3.0.0-release 中都支持 SeaTunnel Engine(Zeta)、Spark 和 Flink。若把每日任务改成 SeaTunnel 作业,并用对应引擎提交,数据可以在 Source → Transform → Sink 之间直接传递,不需要先落盘再交给 Loader。若保留现有 Spark/Flink DAG,SeaTunnel 不会自动接管内存中的 DataFrame 或 Stream,需要改造成 SeaTunnel 作业,或让 Source 读取已有系统中的数据
> HugeGraph Source 和 Sink 在 SeaTunnel 3.0+ 中都支持 SeaTunnel Engine(Zeta)、Spark 和 Flink。若把每日任务改成 SeaTunnel 作业,并用对应引擎提交,数据可以在 Source → Transform → Sink 之间直接传递,不需要先落盘再交给 Loader。若保留现有 Spark/Flink DAG,SeaTunnel 不会自动接管内存中的 DataFrame 或 Stream,需要改造成 SeaTunnel 作业,或让 Source 读取已有系统中的数据

Loader 和 Tools 的优势是专注、直接、上手快。需要直接导入图数据时可先用 Loader;需要备份、恢复、导出或日常运维时可用 Tools。如果已经有 SeaTunnel 作业,通常在原管道中接入 HugeGraph 更方便。需要更高导入吞吐时,Loader 的 bypass-server 和其他导入优化更合适;Loader 在特定后端、数据规模和硬件条件下实测峰值可达 100~200 万条/秒,不能直接当作通用性能承诺,仍需单独压测。新建 SeaTunnel 任务使用 [3.0+](https://github.com/apache/seatunnel/tree/3.0.0-release) 和 `mappings`。使用其他版本时,请重新核对连接器配置。
Loader 和 Tools 的优势是专注、直接、上手快。需要直接导入图数据时可先用 Loader;需要备份、恢复、导出或日常运维时可用 Tools。如果已经有 SeaTunnel 作业,通常在原管道中接入 HugeGraph 更方便。需要更高导入吞吐时,Loader 的 bypass-server 和其他导入优化更合适;Loader 在特定后端、数据规模和硬件条件下实测峰值可达 100~200 万条/秒,不能直接当作通用性能承诺,仍需单独压测。新建 SeaTunnel 任务使用 [3.0+](https://seatunnel.apache.org/download/) 和 `mappings`。使用其他版本时,请重新核对连接器配置。

## 2 准备环境

### 2.1 获取 SeaTunnel 3.0+

SeaTunnel 3.0+ 官方开发文档列出 JDK 8 和 JDK 11;本文统一使用 JDK 11,并设置 `JAVA_HOME`。从 [SeaTunnel 3.0+](https://github.com/apache/seatunnel/tree/3.0.0-release)<sup>[4]</sup> 获取源码,按上游[开发环境文档](https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/developer/setup.md)<sup>[5]</sup>构建发行包:
使用 JDK 11 并设置 `JAVA_HOME`;官方[部署文档](https://seatunnel.apache.org/docs/getting-started/locally/deployment/)
列出的前置要求为 Java 8 或 11。从[官方下载页](https://seatunnel.apache.org/download/)<sup>[4]</sup>
选择 3.0+ 二进制发行包,并使用页面提供的校验文件或签名验证下载包。
下面以 3.0.0 为例,请将 `SEATUNNEL_VERSION` 改为实际下载的发行版本。

```bash
git clone --branch 3.0.0-release https://github.com/apache/seatunnel.git
cd seatunnel
./mvnw clean package -pl seatunnel-dist -am -Dmaven.test.skip=true
export SEATUNNEL_VERSION=3.0.0
tar -xzf "apache-seatunnel-${SEATUNNEL_VERSION}-bin.tar.gz"
cd "apache-seatunnel-${SEATUNNEL_VERSION}"
```

解压 `seatunnel-dist/target/` 中生成的二进制包,后续命令都在解压后的 SeaTunnel 安装目录执行。需要更新功能时,可以切换到其他版本;引擎与连接器插件应来自同一次构建,避免混用不同版本的 JAR。
二进制发行包不包含连接器插件。在 `config/plugin_config` 中选择示例需要的连接器:

本文使用 SeaTunnel 自带的 **Zeta 引擎和 local 模式**<sup>[6][7]</sup>。确认安装目录的 `connectors/` 中包含 HugeGraph,以及所需的 JDBC 或 Kafka 连接器<sup>[11][12]</sup>;如果自定义构建没有包含它们,需补齐同一次构建产出的插件。JDBC 示例还需要将 MySQL 驱动 JAR 放入 `lib/`,驱动类为 `com.mysql.cj.jdbc.Driver`。
```text
--seatunnel-connectors--
connector-hugegraph
connector-jdbc
connector-kafka
--end--
```

按官方[插件安装说明](https://seatunnel.apache.org/docs/getting-started/locally/deployment/#download-the-connector-plugins)<sup>[5]</sup>
安装同版本的已发布插件:

```bash
sh bin/install-plugin.sh "${SEATUNNEL_VERSION}"
```

后续命令均在该安装目录执行,引擎和连接器插件保持相同版本。
本文使用自带的 **Zeta 引擎和 local 模式**<sup>[6][7]</sup>。确认 `connectors/` 中有 HugeGraph,以及所需的 JDBC 或 Kafka 插件<sup>[11][12]</sup>。
JDBC 示例还需将 MySQL 驱动 JAR 放入 `lib/`,驱动类为 `com.mysql.cj.jdbc.Driver`。

### 2.2 准备 HugeGraph 和数据源

Expand Down Expand Up @@ -294,7 +314,7 @@ HugeGraph Sink 是 **at-least-once(至少一次)** 写入,故障恢复可
| `batch_size` | 单批记录数,默认 500 |
| `env.sink.flush.interval` | Zeta 定时刷新间隔,单位毫秒 |
| `check_vertex` | 写边时检查端点,本文的边任务设为 `true` |
| [`batch_failure_fallback`](https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/connectors/sink/HugeGraph.md)<sup>[2]</sup> | 默认 `true`,批量失败后逐条重试,最多跳过 `max_insert_errors` 条失败记录;本文示例显式设为 `false`,让批量失败直接终止任务 |
| [`batch_failure_fallback`](https://seatunnel.apache.org/docs/connectors/sink/HugeGraph/)<sup>[2]</sup> | 默认 `true`,批量失败后逐条重试,最多跳过 `max_insert_errors` 条失败记录;本文示例显式设为 `false`,让批量失败直接终止任务 |
| `max_insert_errors` | 逐条回退时允许跳过的失败记录数;默认 `500`,`-1` 表示不限制,仅在开启 `batch_failure_fallback` 时生效 |

遇到问题时可按下面检查:
Expand All @@ -314,33 +334,25 @@ HugeGraph Sink 是 **at-least-once(至少一次)** 写入,故障恢复可

**HugeGraph 连接器**

<p><sup>[1]</sup> <a href="https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/connectors/source/HugeGraph.md">HugeGraph Source</a><br>
<sup>[2]</sup> <a href="https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/connectors/sink/HugeGraph.md">HugeGraph Sink</a></p>
<p><sup>[1]</sup> <a href="https://seatunnel.apache.org/docs/connectors/source/HugeGraph/">HugeGraph Source</a><br>
<sup>[2]</sup> <a href="https://seatunnel.apache.org/docs/connectors/sink/HugeGraph/">HugeGraph Sink</a></p>

**配置与部署**

<p><sup>[3]</sup> <a href="https://seatunnel.apache.org/docs/introduction/concepts/config/">HOCON 作业文件配置说明</a><br>
<sup>[4]</sup> <a href="https://github.com/apache/seatunnel/tree/3.0.0-release">SeaTunnel 3.0.0-release 分支</a><br>
<sup>[5]</sup> <a href="https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/developer/setup.md">SeaTunnel 开发环境文档</a><br>
<sup>[4]</sup> <a href="https://seatunnel.apache.org/download/">SeaTunnel 3.0+ 正式版下载</a><br>
<sup>[5]</sup> <a href="https://seatunnel.apache.org/docs/getting-started/locally/deployment/#download-the-connector-plugins">连接器插件安装说明</a><br>
<sup>[6]</sup> <a href="https://seatunnel.apache.org/docs/getting-started/locally/deployment/">SeaTunnel 本地部署</a></p>

**执行引擎**

<p><sup>[7]</sup> <a href="https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/engines/overview.md">SeaTunnel 引擎概览</a><br>
<sup>[8]</sup> <a href="https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/engines/spark.md">SeaTunnel Spark 引擎</a><br>
<sup>[9]</sup> <a href="https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/engines/flink.md">SeaTunnel Flink 引擎</a><br>
<sup>[10]</sup> <a href="https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/introduction/concepts/connector-v2-features.md">Connector V2 多引擎说明</a></p>
<p><sup>[7]</sup> <a href="https://seatunnel.apache.org/docs/engines/overview/">SeaTunnel 引擎概览</a><br>
<sup>[8]</sup> <a href="https://seatunnel.apache.org/docs/engines/spark/">SeaTunnel Spark 引擎</a><br>
<sup>[9]</sup> <a href="https://seatunnel.apache.org/docs/engines/flink/">SeaTunnel Flink 引擎</a><br>
<sup>[10]</sup> <a href="https://seatunnel.apache.org/docs/introduction/concepts/connector-v2-features/">Connector V2 多引擎说明</a></p>

**数据源连接器**

<p><sup>[11]</sup> <a href="https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/connectors/source/Jdbc.md">JDBC Source</a><br>
<sup>[12]</sup> <a href="https://github.com/apache/seatunnel/blob/3.0.0-release/docs/zh/connectors/source/Kafka.md">Kafka Source</a><br>
<p><sup>[11]</sup> <a href="https://seatunnel.apache.org/docs/connectors/source/Jdbc/">JDBC Source</a><br>
<sup>[12]</sup> <a href="https://seatunnel.apache.org/docs/connectors/source/Kafka/">Kafka Source</a><br>
<sup>[13]</sup> <a href="https://seatunnel.apache.org/docs/connectors/source/MySQL-CDC/">MySQL CDC Source</a></p>

**旧版本兼容**

<p><sup>[14]</sup> <a href="https://github.com/apache/seatunnel/blob/2.3.13/docs/zh/connectors/sink/HugeGraph.md">SeaTunnel 2.3.13 HugeGraph Sink</a></p>

> **旧版本说明**
>
> 本文面向 SeaTunnel 3.0+,文中的 Source、`mappings` 和图迁移示例不适用于 2.3.13。2.3.13 已过时,仅提供 HugeGraph Sink,配置使用 `schema_config`,并且需要提前创建图模型。如必须使用 2.3.13,请参考[官方 Sink 文档](https://github.com/apache/seatunnel/blob/2.3.13/docs/zh/connectors/sink/HugeGraph.md)<sup>[14]</sup>,不要套用本文配置
Loading
Loading