EventCatalog October 2024 update

October has been a busy month for EventCatalog. The community has grown from 800 Discord members to 856, with 360 new Catalogs created and over 120 organizations actively using EventCatalog.

EventCatalog got a new gold sponsor Gravitee and Hookdeck continues their sponsorship 🙏, this is a great step towards open source sustainability. If you would like to sponsor the project you can check out the tiers on GitHub or contact me directly at dave@eventcatalog.dev.

Our open source community continues to grow with issues and pull requests also increasing. I want to give a personal shout out to Omid Eidivandi and Carlos Rodrigues for their contributions and helping the community with their issues and supporting them, thank you!

We launched new official generators (integrations) with Amazon EventBridge and Backstage and our community have launched new generators for Hookdeck and Atlassian Compass.

A new UI was built for EventCatalog driven by user feedback and improvements were made to existing generator integrations, including the ability to fetch AsyncAPI files from remote urls.

In this blog post I will dive deeper into features that were released in October, why they were built and how they can help you govern your architectures. At the end of the blog post I will share a personal update and what’s planned for November:

⭐️ Updates

• Personal update
• New simplified user interface and user experience for EventCatalog
• Added ability to add query (messages) to EventCatalog
• Simplified folder structure for your EventCatalog
• Embed EventCatalog into your Backstage portals
• Connect Amazon EventBridge to EventCatalog
• Fetch AsyncAPI files from remote URLS and generate your catalogs
• Summary

Personal update

I started EventCatalog 2 years ago as a random idea and side project, understanding that governance with event-driven architecture is a mess and not many solutions are out there to fix that problem. Since then EventCatalog has gained momentum and traction, the project is now at 1.8k stars, over 18k catalogs created and requires more time to maintain and manage.

Since June I have been working on EventCatalog 2 days a week, this has given me time to focus on the project, integrations and build it to what we have today, but with the growth of the project, issues raising, pull requests management it’s becoming hard to maintain. So from November, 2024 I will be spending more time on EventCatalog (from 2 days to 4 days a week) ❤️.

EventCatalog over the past few months has shown me signals of some great potential and growth, many conversations with companies and industry leaders have shown me that there is a space and need for this open source project. I truly believe in the project vision to bring discoverability to event-driven architectures and I think EventCatalog can truly help us.

Using this extra time on the project will let me focus on more automation, community, documentation and project sustainability. The landscape I have identified with EventCatalog is huge, and I’m super excited to see what can be done in the space to make peoples life easier but it comes at one problem….

Having sustainable open source projects is hard.

So over the next few months I will be exploring sponsorships, funding, generator licensing and consulting. So if any of these sound interesting to you or you would like to work with me, feel free to reach out to dave@eventcatalog.dev

I will be reviewing my sustainability plan in Q1 2025 which could even include funding for the project (angels or investors if you are reading, reach out ;)).

For now, I’m focusing on the community and the project. I believe EventCatalog has huge potential and I’m excited to see what we can do in this space.

New simplified user interface and user experience for EventCatalog

We are always looking for ways to improve the user interface and developer experience of EventCatalog. Users have been providing feedback, and we are happy to share some improvements to the catalog.

Improved search functionality for your documentation

You can now search and filter your documentation using the new vertical navigation sidebar. This enables you to quickly find documentation for your domains, services or messages without having to scroll through hundreds of messages.

This functionality has also been added to the visualiser, allowing you to quickly find visuals for your domains, services and messages.

Collapsible sidebar

Sometimes the sidebar can get in the way, so we made it collapsible. Clicking the icon in the vertical navigation bar will open/close the sidebar, giving your room for your documentation.

Minor improvements

Other minor improvements include:

• Added GitHub and Discord links to Catalogs to help folks get help / request features
• Increased space for all pages reducing padding
• Vertical navigation bar introduced for future feature development
• Fixing search bar (cmd K) across the application.

If you have any more UI/UX improvements or requests, please let us know on GitHub or our Discord community.

Added ability to add query (messages) to EventCatalog

EventCatalog consists of many different resource types (domains, services and messages (commands and events)). We are happy to share that you can also document your query (messages) in EventCatalog.

Queries allow you to clearly outline which messages a service accepts and which it invokes. Queries are often seen in protocols such as HTTP and gRPC.

With the support of documenting queries, events and commands, you can now visually represent and document all the messages exchanged between services within your domains using EventCatalog.

; See a query example in the EventCatalog demo →

To learn more you can read the launch blog post, or get started by reading our documentation.

Simplified folder structure for your EventCatalog

We want to make EventCatalog easy to use and be a delightful tool to use for developers, architects and leaders. Previously EventCatalog forced developers to categorise their resources into /domains, /services, /events and /commands folders. When you have many services and hundreds of messages this becomes hard to scale and organise.

EventCatalog now now supports resource subfolders within your EventCatalog projects. This lets you embed domains, services and messages into various different subfolders. This can help you organise your EventCatalog as your catalog grows over time.

my-catalog
├── /domains
│   ├── /Orders
│   │    └──index.md
│   │    └──openapi.yml
│   │    └──index.md
│   │    └──/versioned
│   │         └──0.0.1
│   │         └──index.md
│   │    └──/services
│   │         └── /OrdersService   
│   │                └──index.md
│   │                └──openapi.yml   
│   │                └── /events  
│   │                   └──OrderPlaced
│   │                       └──index.md
│   │                   └── /OrderCancelled   
│   │                           └──index.md
│   └── /Payment
│   │      └──index.md
│   │        └──/services
│   │           └──/PaymentService
│   │               └── index.md
│   │               └── asyncapi.yml
│   │               └── /events
│   │                   └── /PaymentReceived
│   │                       └── /index.md
│   │                       └── /schema.json
│   │                   └── /PaymentCancelled
│   │                       └── /index.md
│   │                       └── /schema.avro
│   │               └── /commands
│   │                   └── /VerifyPayment
│   │                       └── /index.md
│   │                       └── /schema.json
│   │                   └── /CancelPayment
│   │                       └── /index.md
│   │                       └── /schema.json
├── teams
│     └──full-stack.md
│     └──mobile-devs.md
├── users
│     └──aSmith.md
│     └──dboyne.md
│     └──mSmith.md
├── eventcatalog.config.js
├── package.json
└── README.md

To learn more you can read our project structure guide on our documentation, we hope this helps, and if you have any more ideas for improvements feel free to raise a ticket on GitHub.

Embed EventCatalog into your Backstage portals

Backstage is an open source developer portal to help people catalog their software and create an ecosystem of software for internal developers. Backstage provides software templates where your developers can provision new resources or you can build custom resources and plugins on top of their developer portal.

One of the most requested features for EventCatalog was Backstage integration. Companies are using EventCatalog to help them document, govern and visualize their event-driven architecture and want some of the features exposed to backstage.

We are happy to announce that a new EventCatalog Backstage plugin has been developed and is ready to help you embed EventCatalog features directly into your Backstage portal.

Integrating EventCatalog into Backstage gives you the ability to see your event-driven architecture visualization, documentation and messages directly in backstage. Using the plugin you can map your Backstage APIS and Services into EventCatalog services and help your developers quickly find and explore your event-driven architecture within Backstage.

To get started you can read the documentation, watch the video tutorial. or visit the Backstage/EventCatalog demo on GitHub.

This is the first version of the plugin. If you have any feature requests, ideas or problems feel to raise an issue on GitHub or join us on Discord.

Connecting Amazon EventBridge to EventCatalog

Amazon EventBridge is a collection of services (scheduler, event bus and event pipes) provided by AWS to help developers build event-driven architectures.

Since its launch in 2019 Over 1.5 million AWS customers are using EventBridge to build event-driven architectures and applications and they process over 2.6 trillion events per month (impressive!).

EventBridge additionally offers a schema registry. Users can create their own schema registries or EventBridge can version and manage schemas for you using the schema discoverability feature. The registry also provides the ability to upload/download schemas, code binding support but we believe there is much more we can do to empower schema registries, this is where EventCatalog can help.

Using EventCatalog and Amazon EventBridge you can visualize your event-driven architecture, add context to your EventBridge schemas and help your developers understand, download and visualise their architectures.

The EventCatalog EventBridge generator pulls schemas from any schema registry, you can configure custom prefix, suffix or match filters to match your schemas to your EventCatalog services. This gives you the ability to add governance and documentation to your architecture with automation from EventBridge schema registry.

To get started, you install the generator and configure your eventcatalog.config.js file.

eventcatalog.config.js

import path from 'path';
import url from 'url';

const __dirname = path.dirname(url.fileURLToPath(import.meta.url));

/** @type {import('@eventcatalog/core/bin/eventcatalog.config').Config} */
export default {
  // ... other configuration
  },
  generators: [
    [
      '@eventcatalog/generator-eventbridge',
      {
        region: 'us-east-1',
        registryName: 'discovered-schemas',
        services: [
          // Maps exact events to the service
          { id: 'Orders Service', version: '1.0.0', sends: [{ detailType: ['OrderPlaced', 'OrderUpdated'], receives:[{ detailType: "InventoryAdjusted"}]}] },

          // Filter by source (all events that match the source get assigned). This example shows any event matching the source
          // "myapp.orders" will be assigned to the inventory service. The inventory service will publish these events.
          { id: 'Inventory Service', version: '1.0.0', sends: [{ source: "myapp.orders"}], receives:[{ detailType: "UserCheckedOut"}] },


          // This service sends events that match the SchemaName prefixing myapp, and will receive events that end with Payment
          { id: 'Payment Service', version: '1.0.0', sends: [{ prefix: "myapp"}], receives:[{ suffix: "Payment" }] }
        ],

        // all services are mapped to this domain
        domain: { id: 'orders', name: 'Orders', version: '0.0.1' },
      },
    ],
    // Just import all events into the Catalog from a registry
    [
      '@eventcatalog/generator-eventbridge',
      {
        region: 'us-east-1',
        registryName: 'discovered-schemas'
      },
    ],
    // Example using optional credentials
    [
      '@eventcatalog/generator-eventbridge',
      {
        region: 'us-east-1',
        registryName: 'discovered-schemas',
        credentials: {
          accessKeyId: 'X',
          secretAccessKey: 'X',
          accountId: 'X',
        },
      },
    ],
  ],
};

When you run or build your catalog, your schemas will be fetched and stored into the domains or services you specified. EventCatalog understands EventBridge schema versioning and will only download schemas if changes have been made, and version them for you in EventCatalog.

If you are using Amazon EventBridge you can get started by reading the documentation, or watching the video tutorial.

Fetch AsyncAPI files from remote URLS and generate your catalogs

EventCatalog supports integrations with your AsyncAPI specification files. This lets you generate domains, services and messages from your AsyncAPI files. Users of EventCatalog are using this generator to pull in messages from teams across the organization to build a catalog of information that can help them discover services and messages with their architecture.

Previously the AsyncAPI generator only worked with file path support. This means you have to specify the path of the AsyncAPI on disk. Now the AsyncAPI generator supports fetching files from remote URLS.

[
    '@eventcatalog/generator-asyncapi',
    {
      services: [
        // Add AsyncAPI file by public URL
        { path: "https://raw.githubusercontent.com/event-catalog/eventcatalog-asyncapi-example/refs/heads/main/asyncapi-files/payment-service.yml", id: "Payment Service"},
        // Add AsyncAPI file using file system
        { path: path.join(__dirname, 'asyncapi-files', 'fraud-detection-service.yml'), "Fraud Service"}
      ],
      domain: { id: 'payment', name: 'Payment', version: '0.0.1' },
      // Run in debug mode, for extra output, if your AsyncAPI fails to parse, it will tell you why
      debug: true
    },
  ],

This new feature lets you pull AsyncAPI files across many different locations. An example of this would be GitHub repositories. You can fetch AsyncAPI files from many different repositories, and have EventCatalog compile and parse them into a tool that helps you govern, document and discover your event-driven architecture.

To get started you read the documentation.

Summary

October has been a great month, the growth of the project has been incredible, and the feedback so far has been great.

November is the first month I will be spending the majority of my time on the project. This will let me focus on growing the project, speaking with users, and working with users to help them improve their event-driven architecture and governance strategies. If you need any help with your event-driven architecture or governance strategies, I would love to work with you. Please reach out on dave@eventcatalog.dev

At EventCatalog we still have many more features to explore, and problems to solve and I can’t wait to see what’s possible.

If you have any questions, get stuck or want to join our community you can join us on our Discord server.

Until next time!, Dave