1.1.0-Beta

Release notes for Obsrv 1.1.0 Beta.

Obsrv 1.1.0 Beta Release

📅 Release Date: 31st August 2024

🌟 What’s New?

🚀 Core Innovations

🌟 Transaction Data Lake Platform

Revolutionize Your Data Landscape with Obsrv! Introducing our all-new Obsrv Transaction Data Lake Platform, now supercharged with Apache Hudi (v0.15.0).

🔗 Data Connector Framework

🛠️ Easy Connector Setup: Quickly configure connectors and start gathering data from various sources with ease.
🔧 Flexible Framework: Customize and build your own connectors by extending the core framework.
🌟 Introduction to Batch and Stream Connectors: Effortlessly implement a batch connector using Python or Java or Scala and a stream connector with Scala or Java.
🌐 New Source Connectors:
- Kafka 🟪 — Stream real-time data from external Kafka clusters by configuring the Kafka connector with your topic and host details.
- Object Store 🗂️ — Access and manage data from various cloud storage providers such as AWS, Azure, and GCP.
- JDBC 🔗 — Connect to relational databases and efficiently pull data with the JDBC connector.
- Knowlg 📚 — Connect with the Sunbird Knowlg graph events by providing topic and host details.

✨ Real-Time Data Transformation Capabilities

Transform your data effortlessly with advanced features such as encryption, data masking, and JSONata transformations. Learn more about JSONata at jsonata.org.

⚡ Cache Indexing

Our new Redis Cache indexing, powered by a cutting-edge streaming Flink job, ensures your data retrieval for denormalization is faster than ever.

🔍 Schema Validation Enhancement

Stay compliant and future-proof with our upgraded validator, now supporting Draft-12 of the JSON Schema specification.

🛠️ Table Schema Modifications

We’ve refined our table schemas to seamlessly integrate new features, making your data management smoother and more efficient.

🔐 Enhanced Security Measures

Critical security vulnerabilities, including SQL injection issues, have been addressed to significantly strengthen the security of our platform.

🌐 API Enhancements

📊 Dataset Management APIs

📁 CRUD Operations: Full control at your fingertips — Create, Read, Update, and Delete your datasets with ease.
🔄 Status Transition: Effortlessly manage and transition dataset statuses.
🔎 Health Monitoring: Comprehensive health monitoring APIs.
📤 Export & Import: Export and import dataset configurations seamlessly across environments.
🌀 Copy & Reset: Simplify dataset management with powerful copy and reset functionalities.
⚙️ Schema Generation: Automatic schema generation via API.
📋 Dataset Listing: Easily retrieve a comprehensive list of all available datasets.

🔌 Connector APIs

🗂️ Read & List Connectors: Manage and explore your connectors effortlessly.

📋 Schema and Event Validation API

Ensure your data’s integrity with powerful APIs for validating schemas and events against established standards.

🧩 Template APIs

🛠️ Template Management: Full CRUD operations for templates.
🔍 Template Query: Query data from various sources using our robust template-driven API.

📈 Data APIs

🔎 Data Query: Access and query your data using SQL or native formats.
📥 Data Ingestion: Bring your data into the system with our streamlined ingestion API.
📤 Data Exhaust: Retrieve and manage your data from the object store.

🚨 Alert and Notification APIs

🔔 Custom Alerts: Configure custom alerts tailored to your needs.
📢 Notification Channels: Configure Slack, email, and Discord notification channels.

🔧 Obsrv Management Console

🚀 Effortless Dataset Management — Manage datasets throughout their entire lifecycle: Draft, Ready for Publish, Publish, and Retire.
🔗 Advanced Connector Configuration — Seamlessly configure Source Connectors and Sink Connectors with a few clicks.
🔄 Seamless Export & Import — Transfer datasets between environments.
⚙️ Automatic Schema Generation & Indexing — Instantly generate schemas and index data across Redis, Druid, Lakehouse, and more.
🚨 Proactive Alerts & Health Monitoring — Configure smart alerts and monitor health of datasets and connectors.
📊 Comprehensive Resource Monitoring — Monitor system performance with advanced resource monitoring tools.
🔍 Powerful Data Querying & Analysis — Run SQL queries, explore data in native formats, and generate reports.

🔧 Automation Upgrades

🚀 Enhanced Automation Scripts

☁️ Cloud & On-Premises Support: Expanded support across AWS, GCP, Azure, and on-premises environments.
🔧 Simplified Installation: Deploy with confidence using our newly refined, easy-to-use installation scripts.

🚫 Deprecation Notice

⚠️ Deprecation of v1 APIs: APIs from version 1.0.x will no longer be supported in this release. We recommend transitioning to the new APIs.

🔄 Migration Guide

🔧 How to Migrate from v1.0.x to v1.1.0: Detailed migration instructions will be added soon.