We're excited to share a batch of new features in Livebook Teams to help you manage access and keep your deployed apps organized.

Here's what's new.

Google Workspace integration for authentication and access control

Control access to your Livebook apps and servers using your existing Google Workspace groups.

Manage access right from Google Workspace. Add someone to a group, and they instantly get access to the right apps in Livebook Teams.

To get started, set up Google Workspace as your SSO provider in Livebook Teams, then map your Google groups to control access at the app server or app level.

Check the docs for setup instructions.

Redesigned apps page with folders and search

The new deployed apps page helps you stay organized as your collection of internal tools grows.

Organize apps into folders, search by name, and filter by folder. Quickly find the runbook or admin tool you need without scrolling through long lists.

Access control by app folder

Control who can access apps in each folder once you've organized them.

For example, give your ops team access to a "production-runbooks" folder, while your support team can use a "support-tools" folder.

Combine this with Google Workspace integration to map folders to Google groups for seamless access management.

Open notebooks from your Git repository

Open notebooks directly from your private Git repository inside your Livebook app server.

This is great for template notebooks and runbooks you version in Git. Your team can browse and open them straight from the repository, no manual imports needed.

Audit logs documentation 

We now have full documentation on audit logs, showing how to track who ran what code and when.

Audit logs are essential for production. When someone runs a runbook that changes production data or system state, you'll have a clear record of who did it and when.

Try it out

To start using these features, update to Livebook v0.18 or newer and configure them in your Teams organization settings.

Livebook v0.14 is here with new features that make navigating your code easier than ever.

No more endless scrolling through a notebook to find module or function definitions — Livebook v0.14 has you covered.

Module definitions in the Outline panel

When you define an Elixir module in a code cell, it will now be listed in the Outline panel in the sidebar.

The Outline panel, previously known as the Sections panel, now gives you a structured view of your code. It lists all the modules defined within each section of your notebook.

You can also use this panel to navigate to module definitions. Just click on a module's name on the Outline, and your notebook will automatically scroll to the position where it is defined.

Here's a video showing how it works:

Want to see how we built this? Here's the pull request.

Go to definition of modules and functions

Not only can you quickly navigate to the definition of a module, but you can also quickly go to the definition of a function.

To do that, hover over a module or function you defined in your notebook, hold Command ⌘ (for macOS) or Ctrl (for Windows/Linux), and click on the module or function call.

Here's a video showing how it works:

You can also access this feature through the link shown in the intellisense for a module or function call:

Want to see how we built this? Here are the pull requests: #2730, #2741.

Conclusion

We're dedicated to continually improving the Livebook developer experience, and these new features are another step in that journey.

Feel free to check out the v0.14 changelog to discover all the features and changes in this newest release.

Happy hacking!

We are thrilled to announce the release of a new version of Livebook!

This version includes numerous new features and improvements. Today, we will highlight one of the most significant additions: Kino.Proxy.

By using Kino.Proxy, you can now expose API endpoints directly from your notebook or Livebook app.

Let's take a look at how it works.

How to expose an API endpoint from your Livebook notebook

Here's a simple "hello world" example:

Kino.Proxy.listen(fn conn ->
    Plug.Conn.send_resp(conn, 200, "hello world!")
end)

The Kino.Proxy.listen/1 function makes Livebook expose API endpoints and will proxy requests to the handler function passed to it.

That handler function receives a Plug.Conn as an argument and should use the Plug API to send a response to the client.

Let's see a video with a simple example working: 

Leveraging Plug for APIs with Kino.Proxy

You can also provide a module plug as an argument to Kino.Proxy.listen/1, like this:

defmodule MyPlug do
  def init([]), do: false

  def call(conn, _opts) do
    Plug.Conn.send_resp(conn, 200, "hello world!")
  end
end

Kino.Proxy.listen(MyPlug)

Since our API handler is a plug, we can leverage other plugs. For example, here's how to use Plug.Router to handle multiple endpoints:

defmodule ApiRouter do
  use Plug.Router

  plug :match
  plug :dispatch

  get "/hello" do
    send_resp(conn, 200, "hello from router")
  end

  get "/echo/:message" do
    send_resp(conn, 200, String.upcase(message))
  end

  match _ do
    send_resp(conn, 404, "oops, not found")
  end
end

Kino.Proxy.listen(ApiRouter)

Livebook apps + Kino.Proxy = handling requests from external apps

This new feature enables an exciting new possibility: a Livebook app can now use Kino.Proxy to handle external HTTP requests, allowing code execution triggered by an external system.

Let’s see an example.

We built a Livebook app that tracks how many days we don't have a broken build in a GitHub repository:

Whenever a new build finishes in the configured Github Repo, GitHub sends a webhook to the Livebook app, which processes the request to update its state.

To handle webhooks inside the Livebook app, we could combine Plug.Router with Kino.listen/1 like this:

defmodule ApiRouter do
  use Plug.Router

  plug(:match)
  plug(Plug.Parsers, parsers: [:json], json_decoder: Jason)
  plug(:dispatch)

  post "/webhook" do
    process_webhook(conn.body_params)

    conn
    |> put_resp_content_type("application/json")
    |> send_resp(200, ~s({"message": "ok"}))
  end

  match _ do
    conn
    |> put_resp_content_type("application/json")
    |> send_resp(404, ~s({"message": "not found"}))
  end

  defp process_webhook(webhook_payload) do
    # Logic to do something with the webhook payload
  end
end

Kino.Proxy.listen(ApiRouter)

Here's a quick video with a demo of that app:

Here's the source code for that Livebook app so you can understand how it works under the hood. 

If you want to run the app by yourself, click here.

Wrapping up

We believe that APIs with Kino.Proxy open Livebook to a new category of use cases.

For example, you can quickly build an API that receives a request from Zapier to automate internal processes.

We're looking forward to seeing what you will build it with!

One last thing: Livebook 0.13 has plenty more, and you can view all of them in our changelog.

We plan to share quicker demos of other Livebook 0.13 features on our X/Twitter, so feel free to follow us there.

Happy hacking!

We're thrilled to announce Tigris as the latest Livebook sponsor!

Tigris is a globally distributed S3-compatible object storage service that provides low latency anywhere in the world.

We've prepared a short demo showcasing the Tigris API through Livebook (of course 😉) to highlight its capabilities.

Quick intro to Tigris

You'll find Tigris intuitive if you're familiar with other object storage services like AWS S3. Here's an example of saving an object to a Tigris bucket:

ExAws.S3.put_object("my_tigris_bucket_name", "cat.jpeg", image)
|> ExAws.request!()

Notice we're using an AWS S3 client. Thanks to Tigris's S3-API compatibility, transitioning your code requires minimal changes.

Let's say the object is uploaded to a public bucket; to retrieve it, it's just a simple HTTP GET:

object_url = "https://fly.storage.tigris.dev/my_tigris_bucket_name/cat.jpeg"
Req.get!(object_url)

Simple, right? Now for the real game-changer…

Imagine uploading a file from São Paulo (Brazil). The object is stored automatically in the nearest Tigris region, GRU (São Paulo).

When a user in Mumbai (India) requests the object, it's initially served from GRU. However, Tigris automatically caches the object in the region closest to the request's origin, significantly improving subsequent access times — all without extra configuration!

Here are some real numbers for that example:

Tigris combines object storage with CDN capabilities. That means great UX for end-users and great DX for developers.

Let's look at a demo notebook that showcases this.

Demo

In our demo, we use Livebook's integration with MapLibre to plot the HTTP request and response locations on a map. We also log the response times in a table.

Additionally, we leveraged the Livebook Apps feature to visualize the notebook as an app.

Here's the result:

Wrap-up

Tigris brings some cool innovations to the object storage space. For example, here's how you could use it as a global key-value store from Elixir.

If you want to try using Tigris with Elixir, they have official Elixir support.

Welcome to the 5th day of Livebook Launch Week 2! 🎉

For those of us who found ourselves instinctively trying to use Vim or Emacs shortcuts in Livebook, only to realize that these weren’t supported, feeling frustrated. We hear you.

We’re excited to announce that now you can use Vim and Emacs key bindings while coding in Livebook.

Let’s see how it works.

Using Vim or Emacs key bindings in Livebook

Go to the settings page in Livebook and choose the “Key bindings” you prefer:

Once that’s done, you can use it in any Livebook cell with a code editor, like in a code cell or a markdown cell. Here’s an example using Vim keybindings:

From now on, you can bring your Vim/Emacs muscle memory to Livebook for a more pleasant and productive experience. Thanks to Kenichi Nakamura, who contributed that feature.

What now?

To start using that new feature, install the latest version of Livebook and have fun!

More of Launch Week 2

• Day 1: Remote execution Smart cell
• Day 2: Speech-to-text with Whisper: timestamping, streaming, and parallelism, oh-my!
• Day 3: Introducing File Integration
• Day 4: Integration with SnowFlake and Microsoft SQL Server

And if you want to discover everything that was added/changed in this release, here’s the changelog.

Welcome to the 4th day of Livebook Launch Week 2! 🎉

Today’s post is about our two new database integrations: Snowflake and SQL Server.

Together with those new ones, Livebook now comes with built-in integrations for seven databases and data warehouses:

• PostgreSQL
• MySQL
• SQL Server
• SQLite
• Google BigQuery
• Amazon Athena
• Snowflake

Let’s see how the new integrations work.

Connecting to Snowflake

The Snowflake integration is available through the Database Connection Smart cell. All you need to do is configure the Smart cell with your connection credentials, and you’ll be ready to start executing queries.

Let’s see how that works.

Under the hood, this is different from the other database integrations we already have. Livebook connects to Snowflake through ADBC (Arrow Database Connectivity), using the ADBC hex package, which contains ADBC bindings for Elixir.

The Arrow format is highly efficient, and it’s also integrated with Explorer, so now you can easily query a database and load the result into an Explorer dataframe.

Good news for those of us doing data analysis with Elixir and Livebook. 🎉

Connecting to Microsoft SQL Server

Connecting to SQL Server is also super simple. All you need to do is add a new Database Connection Smart cell, select SQL Server, and fill in your database credentials.

Here’s how it works.

This was a community contribution from Simon McConnell. Curious fact: the pull request contains less than 250 LOC! Cool, right?! 😎

What now?

To start playing with those new features, install the latest version of Livebook and have fun!

More of Launch Week 2

• Day 1: Remote execution Smart cell
• Day 2: Speech-to-text with Whisper: timestamping, streaming, and parallelism, oh-my!
• Day 3: Introducing File Integration
• Day 5: Vim and Emacs key bindings

Welcome to the 3rd day of Livebook Launch Week 2! 🎉

Today, we’re excited to announce Livebook v0.11’s most prominent feature, file integration.

Although file integration may not sound exciting at first glance, bear with us; we have some elegant features to show.

Imagine code generation when you drag and drop a file based on the file type. Or quickly do a lazy reading of a multiple gigabyte parquet file stored in S3. That’s what we’re talking about! 😎

Watch the video where José Valim shows a demo of what this feature can do.

Why?

Throughout the last releases, we have been improving how Livebook helps to work with data.

Data comes from different sources. One common source is a database, which Livebook already has good support for. Another source is a file, either on your machine or on S3, which Livebook hasn’t natively integrated with yet.

Don’t get us wrong, you could already write some Elixir code in your notebook that reads a file from the file system or the web. But the Livebook UI had no understanding of it. And that’s what changed in this release.

What’s the Livebook file integration?

The concept is simple. When you’re editing a notebook, you can add a file to it. All files are listed in the sidebar, so everyone using your notebook can quickly glance at your file dependencies. Once you do that, Livebook exposes an API for your notebook to read that file via the Kino.FS module.

But that’s not the only thing. Now that Livebook natively understands files, it can offer code suggestions based on the file type.

For example, when you drag and drop a CSV file to Livebook, it can generate code that creates an Explorer Dataframe from that data:

Or when you drag and drop a SQLite database file, Livebook can generate code that describes that database:

Or when you drag and drop an audio file, Livebook can generate code that uses a Machine Learning model to generate a transcription for you:

One thing you’ll notice in common with all those drag-and-drop examples is that Livebook doesn’t automatically execute the code. Instead, it emits the code and allows you to run it yourself.

This may look like a small detail, but it’s part of our vision for Livebook. We don’t want it to be a magic box that executes some task behind the scenes that you can’t know what it is. We want to enable you to introspect the code, learn from it, and customize it to your needs.

We took that approach with Smart cells. And that’s the approach we’re continuing to apply.

File references and file attachments

When you start adding files to your notebook, you’ll notice that there are two types of files: References and Attachments.

References are files that point to existing resources on your disk, a remote storage, or a URL. It’s like a symbolic link; Livebook doesn’t store the file, it just has a pointer to it.

Attachments are different. They are files stored in the files/ directory alongside your notebook source. So, a file attachment will be kept wherever you save your notebook.

So, if you’re storing a file in an S3-compatible cloud service, in a web server, or if you want to save it and version it alongside your notebook source, Livebook has you covered.

Lazy data reading and manipulation

When you have a dataset with multiple gigabytes of data, you don’t want to load all of it at once to the memory. A common approach for that is lazy loading the data. And this new Livebook release supports that workflow as well.

For example, let’s say your dataset is stored as a Parquet file in S3. When you add that file to your notebook, Livebook will give the option to load that data into a dataframe lazily:

Not only that, but you can also lazily manipulate that data:

When needed, Livebook will download all the data, like when plotting a chart based on the data or when exporting it after some data wrangling:

When we announced Bumblebee, a collection of pre-trained models inspired by Hugging Face Transformers, the Whisper speech-to-text model quickly became one of the favorite and most used models within the Elixir community.

Thanks to advancements in the overall Numerical Elixir ecosystem, Livebook v0.11 includes a highly improved integration with Whisper, which we will detail in this article.

If you want to skip ahead and give it a try, install Livebook and start a new notebook. Then click “+ Smart cell” and choose “Neural Network task.” You will find Whisper as Speech-to-text under Audio.

New features

There are three new features in our Whisper integration:

1. Timestamping: we now include timestamps on audio segments.
2. Streaming: our previous version of Whisper was limited to 30 seconds of audio, leaving it up to users to break their audio apart. This new version is capable of streaming both inputs and outputs. You can give arbitrarily long files to the model, which will be streamed as input, and the model will proceed to merge and stream transcriptions as they arrive.
3. Parallelism: in addition to streaming, files with more than 30 seconds will be split and batched according to the Neural Network batch size. For example, with a batch size of 10, up to 5 minutes of audio can be processed in parallel. Thanks to this, we expect our models to perform inference an order of magnitude faster compared to Open AI’s implementation when transcribing larger files on the GPU.

Of course, all of those features work together, providing a delightful experience as you can see below, where we transcribe on the fly one of Thinking Elixir episodes:

When you combine the features above with Nx’s ability to run neural networks distributed across multiple machines and GPUs, Elixir developers now have a first-class, state-of-the-art, speech-to-text model ready to run, enjoy, and scale.

What now?

Try for yourself!

Transcribe an audio file using our built-in Neural Network Task Smart cell. Maybe start with a small file to quickly see the result. Then you can try a bigger one.

Download the latest Livebook version and have fun!

More of Launch Week 2

• Day 1: Remote execution Smart cell
• Day 3: Introducing File Integration
• Day 4: Integration with SnowFlake and Microsoft SQL Server
• Day 5: Vim and Emacs key bindings

Welcome to the second Livebook Launch Week! 🎉

If this is your first Launch Week with us, let us explain how this works. Starting today, each day of this week, we’ll announce a new feature of this release, Livebook v0.11.

Today, we’ll discuss our newest built-in Smart cell: Remote execution.

Calling code from other nodes before the new Smart cell

Livebook is a multi-purpose tool. And two of the most common use cases we’ve been seeing are debugging and internal tools.

A common requirement for both use cases is to write code that calls functions from a remote node, probably your main Elixir app in the production environment.

One way to do that is by connecting your notebook to the other node using the “Attached node” runtime:

That said, that approach has a limitation. When you use it, all of the code in your notebook will run in the context of that remote node. So, you can’t add Mix dependencies to your notebook, like VegaLite for charts or Kino for user interactions.

The other way we have to run remote functions is by using the default runtime (Elixir standalone) and using the erpc Erlang module, like this:

But, we noticed people have been using the “Attached node” by default when calling a function from a remote node, even when they need to add dependencies to their notebook. That’s a problem because although the “Attached node” setting is powerful for debugging, we believe that most of the time you need to interact with a remote node, you should use the erpc approach.

So, we decided to make the erpc approach a first-class citizen in Livebook. It’s now available as a new Smart cell that will streamline the process of calling functions from another node.

Calling code from other nodes using the Remote execution Smart cell

Here’s how the new Smart cell works:

As with any other Smart cell, one of the things it helps with is generating boilerplate code for you. There is no need to write the code that connects your notebook to a remote node over and over again.

But it goes beyond that.

The Smart cell can store the remote node’s cookie as a Livebook secret, so you don’t need to hardcode sensitive values inside your notebook. And with the upcoming Livebook Teams, it will be easy to share this secret with everyone on the team in a secure manner.

Last, this new Smart cell gives you autocompletion and docs preview of the modules defined in the remote node! Isn’t that awesome?! 🤯

We hope this feature will simplify the development of internal tools that need to communicate with existing nodes.

As an example, here’s a video showing how to use the new feature to connect a Livebook notebook to a Phoenix app, get some data from it, display a chart with those metrics, and share that as an interactive Livebook app:

What now?

All of the new features of this new release are already available today!

To start playing with them, install the latest version of Livebook and have fun!

More of Launch Week 2

• Day 2: Speech-to-text with Whisper: timestamping, streaming, and parallelism, oh-my!
• Day 3: Introducing File Integration
• Day 4: Integration with SnowFlake and Microsoft SQL Server
• Day 5: Vim and Emacs key bindings

We’re happy to announce that we’re sponsoring Spawnfest for the second time!

SpawnFest is an annual 48-hour online software development contest in which teams worldwide get one weekend to create the best BEAM-based applications they can.

We are collaborating with them again to create a bracket of the event dedicated to Livebook projects. Last year, most of the entries involved Kino and Smart Cells, but this year we are changing things up. Our Spawnfest bracket this year will focus on Livebook Apps.

Livebook Apps is a feature we launched earlier this year that enables you to turn your notebook into an interactive web application.

What you can build with Livebook Apps

Since Livebook is a general-purpose tool, there aren’t many constraints on what you can build with Livebook Apps. So, it’s up to your imagination. But here are some examples to give you some inspiration.

A multi-user real-time app

Livebook comes with real-time collaboration out of the box. You can use that power in your Livebook apps as well.

Here’s a video showing how to build a chat app.

A machine learning app

Livebook integrates with Hugging Face to bring you pre-trained models and helps you to use them with just a few clicks.

Here’s a video showing how to build an app that uses the Whisper machine learning model to transcribe audio messages in a chat.

A data visualization app

Livebook integrates with Vegalite and Maplibre to make it super easy to build lots of different kinds of data visualization.

Here’s an example of an app that plots a chart with the number of stars a Github project got over time.

This app is deployed, so you can try it live. And here is the source code.

A workflow automation app

One month ago, we expanded Livebook Apps with a feature called Multi-Session Livebook Apps. We believe that this new feature is excellent for the automation of technical and business workflows.

You can think of it as a way to transform a script into a UI and share that with others by just sharing an URL.

Here’s a video showing how to build an app that gets data from Github’s API, generates a report, and sends it to a Slack channel.

How to deploy a Livebook App

Livebook Apps run inside a Livebook instance. So, you can run them in any Livebook installation, be it in the localhost or the cloud.

One of the easiest ways to install Livebook in the cloud is using Docker. Let’s see two examples of deploying a Livebook App this way.

How to deploy a Livebook App to Fly.io

First, make sure you have Fly’s command-line installed on your machine.

Now, clone the following template repo:

git clone https://github.com/hugobarauna/livebook-apps-on-fly-template.git my-livebook-apps

Add a file with the source code of your Liveobok App to the public-apps/ directory of your repository.

Then, follow Fly’s instructions to Deploy via Dockerfile.

After that, Livebook and your Livebook App will be running inside Fly.

Here’s a video showing how that works.

How to deploy a Livebook App to Hugging Face

To deploy a Livebook App to Hugging Face, we’ll use Hugging Face Docker Spaces.

First, install Livebook on Hugging Face by following these instructions.

Second, add a file with the source code of your Liveobok App to the public-apps/ directory of your Space and make a commit.

After that, Hugging Face will rebuild your Space, and your Livebook app will be deployed.

Here’s a video showing how that works.

How to participate in Spawnfest

Participation in SpawnFest is 100% free of charge. To register, go to their website and follow the instructions.

We’re looking forward to seeing what you will build with Livebook Apps!