Skip to main content

What is Legend?

Legend is an open-source data platform. You describe your data once, as a model, and connect that model to where the data lives. Then you query, share and publish the data in the model's terms instead of the stores'. Legend was built at Goldman Sachs and contributed to FINOS in October 2020.

How Legend works​

  1. Model the data. A model names the business concepts, such as a trade, a firm or a person, with their properties and how they relate to each other. It also states the rules the data must follow. Models are written in the Legend language, so they are code: kept in a project, reviewed, versioned and tested.
  2. Connect the model to the data. A mapping ties the model to a store: a relational database such as Snowflake, Databricks, BigQuery or Postgres, files in formats such as JSON, XML or CSV, or HTTP services. A runtime says which connection to use for each store.
  3. Query the data. Queries are written against the model. The engine compiles each query into an execution plan, runs it against the stores, for example as SQL, and returns the results in the model's terms. Tabular data can also be queried directly with the Relation API or with Legend SQL.
  4. Share the data. A query can become a service, which other applications call as an API. A function can be activated to run elsewhere, for example as a Snowflake app. A data product publishes data for others to find and request access to.

Because the model, the mappings and the tests live together, everyone who uses the data sees the same definitions, and a change is reviewed and tested before anyone depends on it.

Applications​

ApplicationWhat it's for
Legend StudioEdit a project: its models, mappings, connections, services, data products and tests. Review changes, then release a version.
Legend QueryBuild and save queries with a drag-and-drop query builder, starting from a data space, a data product or a service.
Legend DataCubeExplore query results in a grid: filter, sort, group, pivot and add columns.
Legend AnalyticsBuild a query as a chain of steps on a canvas, with Legend Cube.
Legend MarketplaceFind data products and request access to them.
Legend REPLRun Legend language expressions from a terminal, and see their results in DataCube.
Legend Pure IDEWrite and run Legend language code against the Pure runtime. It is used to develop the language itself.

Servers​

ServerWhat it does
Legend EngineCompiles models, generates execution plans and runs them against stores. It also runs services and activates functions.
Legend SDLCKeeps each project in a GitLab repository, with workspaces, reviews and released versions.
Legend DepotStores released versions of projects, so that projects can depend on each other and applications can query them.

The code for all of them is on GitHub: legend-studio for the applications, and legend-engine, legend-sdlc, legend-depot and legend-pure for the servers and the language.

Where to go next​