Visual Flow, powered by Apache Spark is a cloud-native open-source ETL tool based on Apache Spark unified analytics engine with a Graphical User Interface. It combines the best attributes of Kubernetes, Spark, and Argo into a single system with the following features:

Portability, flexibility and multi-cloud compatibility Increased developer productivity High availability, performance and fault tolerance Cost-effective Open-source

Visual Flow provides a Graphical User Interface to create ETL jobs and combine them in Data Processing Pipelines, run/schedule them, and monitor execution. It is a multi-tenant system. You can create multiple projects in a single cluster. Each project is represented as a namespace, which allows allocating resources and providing access to the users in different roles.

Visual Flow does not require a database. It is possible to create all objects as native Kubernetes resources. Our tool leverages Kubernetes authorization to manage users and their roles within a project.

Visual Flow provides the ability to create parameters (e.g., connection info) and reuse them in Jobs.

Why Visual Flow?

Graphical User Interface speeds up learning and jobs development Good monitoring and alerting capabilities out of the box Error Handling out of the box Organizing jobs into Data pipelines Unlimited parallelism Unlimited scalability Easy to customize and extend Easy to test automatically Can be installed on-premise as well as on cloud provider of your choice Can be packaged and provided as SaaS solution on your own Cloud platform