English | 中文

Apache IoTDB Extras is a companion repository to the main Apache IoTDB project. It contains examples, integration modules, connectors, and additional tools that extend the functionality of IoTDB.

IoTDB (Internet of Things Database) is a time series database designed specifically for the Internet of Things (IoT) scenarios, with high performance for data ingestion, storage, and querying for time series data.

This repository provides integrations with popular data processing frameworks, visualization tools, and other systems to enhance IoTDB's ecosystem.

This repository includes:

1. Connectors: Integration modules for various data processing frameworks and systems

   • Flink IoTDB Connector
   • Flink SQL IoTDB Connector
   • Flink TsFile Connector
   • Grafana Connector & Plugin
   • Hadoop Connector
   • Hive Connector
   • Spark IoTDB Connector
   • Spark TsFile
   • Zeppelin Interpreter

2. IoTDB Collector: Data collection framework for IoTDB

3. IoTDB Spring Boot Starter: Spring Boot integration for IoTDB

4. Kubernetes Support

   • Helm Charts
   • IoTDB Operator

5. MyBatis Generator: Code generation tools for database access

6. Metric Scrape: Standalone Prometheus text exposition scraper for the IoTDB table model

7. Examples: Sample applications and code examples demonstrating the use of IoTDB with various technologies

• Java 8+ (JDK 1.8 or later versions, springboot requires JDK 17+, Recommended JDK 17+)
• Maven 3.6 or later
• Git

To build the project from source, follow these steps:

1. Clone the repository:

   git clone https://github.com/apache/iotdb-extras.git
   cd iotdb-extras

2. Build the project with Maven:

   # Build the entire project (includes distributions,iotdb-collector,metric-scrape,mybatis-generator)
   mvn clean package -DskipTests
   
   # Or build all
   mvn clean package -DskipTests -Pwith-all-connectors,with-examples,with-springboot,with-grafana
   

3. The build artifacts will be located in the target directory of each module.

IoTDB-Extras uses Maven profiles to configure different build options. You can combine various profiles to build specific components:

• with-springboot: Build Spring Boot integration

  mvn clean package -Pwith-springboot -DskipTests

• with-examples: Build example applications(Recommended for use in conjunction with Connector Profiles)

  mvn clean package -Pwith-examples -DskipTests

• with-all-connectors: Build all available connectors (includes Flink, Hadoop, Hive, Spark, Zeppelin. excludes Grafana)

  mvn clean package -Pwith-all-connectors -DskipTests

• with-spark: Build Spark connectors (includes spark-iotdb-connector, spark-iotdb-table-connector)

  mvn clean package -Pwith-spark -DskipTests

• with-flink: Build Flink connectors (includes flink-iotdb-connector, flink-sql-iotdb-connector, flink-tsfile-connector)

  mvn clean package -Pwith-flink -DskipTests

• with-grafana: Build Grafana connectors and plugins

  mvn clean package -Pwith-grafana -DskipTests
  
  # (Recommend) If you want to build the Grafana plugin, use:
  mvn clean package -Pwith-grafana-plugin -DskipTests
  
  # To build the Grafana connector, use:
  mvn clean package -Pwith-grafana-connector -DskipTests
  

• with-hadoop: Build Hadoop connector

  mvn clean package -Pwith-hadoop -DskipTests

• with-hive: Build Hive connector

  mvn clean package -Pwith-hive -DskipTests

• with-spark-tsfile: Build Spark TsFile connector (includes spark-tsfile, hadoop)

  mvn clean package -Pwith-spark-tsfile -DskipTests

• with-zeppelin: Build Zeppelin interpreter

  mvn clean package -Pwith-zeppelin -DskipTests

When building the examples module, you can use these profiles:

• with-springboot: Build Spring Boot examples

  mvn clean package -Pwith-examples,with-springboot -DskipTests

• with-flink: Build Flink examples

  mvn clean package -Pwith-examples,with-flink -DskipTests

• with-hadoop: Build Hadoop examples

  mvn clean package -Pwith-examples,with-hadoop -DskipTests

• with-spark: Build Spark examples

  mvn clean package -Pwith-examples,with-spark -DskipTests

You can combine multiple profiles to build specific components:

# Build Flink and Spark connectors with examples
mvn clean package -Pwith-flink,with-spark,with-examples -DskipTests

# Build a complete distribution with all components
mvn clean package -Pwith-all-connectors,with-examples,with-springboot -DskipTests

This repository includes a variety of examples demonstrating how to use IoTDB with different technologies:

• Function: Demonstrates how to send data to an IoTDB server from a Flink job
• Usage:
  • Launch the IoTDB server
  • Run org.apache.iotdb.flink.FlinkIoTDBSink.java to run the Flink job on local mini cluster

• Usage:
  • Run org.apache.iotdb.flink.FlinkTsFileBatchSource.java to create a TsFile and read it via a Flink DataSet job
  • Run org.apache.iotdb.flink.FlinkTsFileStreamSource.java to create a TsFile and read it via a Flink DataStream job
  • Run org.apache.iotdb.flink.FlinkTsFileBatchSink.java to write a TsFile via a Flink DataSet job
  • Run org.apache.iotdb.flink.FlinkTsFileStreamSink.java to write a TsFile via a Flink DataStream job

• Examples include:
  • BatchSinkExample.java: Demonstrates batch writing to IoTDB
  • BoundedScanExample.java: Shows bounded data scanning
  • CDCExample.java: Illustrates Change Data Capture
  • LookupExample.java: Shows lookup functionality
  • StreamingSinkExample.java: Demonstrates streaming data writing

• Introduction: Demonstrates how to use the IoTDB-Table-Spark-Connector to read and write data from/to IoTDB in Spark
• Version:
  • Scala 2.12
  • Spark 3.3 or later
• Usage:

  <dependency>
      <groupId>org.apache.iotdb</groupId>
      <artifactId>spark-iotdb-table-connector-3.5</artifactId>
  </dependency>

• Configuration Options:

  Key	Default	Description	Required
  iotdb.database	--	Database name in IoTDB	true
  iotdb.table	--	Table name in IoTDB	true
  iotdb.username	root	Username to access IoTDB	false
  iotdb.password	root	Password to access IoTDB	false
  iotdb.urls	127.0.0.1:6667	Connection URLs	false

• Introduction: Shows how to use iotdb-spring-boot-starter
• Version:
  • IoTDB: 2.0.3
  • iotdb-spring-boot-starter: 2.0.3
• Setup:
  1. Install IoTDB
  2. Create necessary database and tables
  3. Configure Spring Boot application

• Function: Shows how to send data from localhost to IoTDB through Kafka
• Version:
  • IoTDB: 1.0.0
  • Kafka: 2.8.0

• RocketMQ Integration
• RabbitMQ Integration
• Pulsar Integration

• Introduction: Shows how to use IoTDB-Mybatis-Generator
• Version:
  • IoTDB: 2.0.2
  • mybatis-generator-plugin: 1.3.2
• Setup:
  1. Install and start IoTDB
  2. Create necessary database and tables
  3. Run the generator

For detailed usage instructions, please refer to the README files in the specific example directories:

• Flink Examples
• Spark Table Examples
• MyBatis Generator Examples
• IoTDB Spring Boot Starter Examples
• Kafka Examples

You can also refer to module-specific documentation:

• Flink IoTDB Connector
• Grafana Plugin
• IoTDB Spring Boot Starter
• Metric Scrape
• Kubernetes Helm Charts
• IoTDB Operator

IoTDB components can be run using Docker. For more information, see the Docker documentation.

Apache IoTDB Extras is released under the Apache 2.0 License.

• Apache IoTDB User Group: 659990460

• Add friend: apache_iotdb, and then we'll invite you to the group.

• Slack channel

see Join the community for more!