• Info Panel

Overview

NeuroNLP++ is a query interface for navigating fruit fly brain datasets. In response to free-form English queries, such as
"what neurons respond to carbon dioxide?",
NeuroNLP++ links the Hemibrain dataset with the worldwide fruit fly brain literature. The linking enables the exploration of the Hemibrain dataset using cell type and general descriptors of function.

It further enables in-depth investigations of the connectomic/synaptomic structure of the fly brain, using intuitive rule-based English queries, such as
"show EPG neurons that have outputs in PB glomerulus L8 or PB glomerulus R2".

For more information about the NeuroNLP++ user interface

Open Overview!

Credits

The original Hemibrain dataset is provided by the HHMI Janelia Research Campus and can be accessed here. It is licensed under CC-BY.

News

Updates and announcements for NeuroNLP++.Hemibrain.

FFBO News See What's New!

Quick Demos

Want to get started? Go check out some real-time "Demos" for NLP queries and GUI usage!

Try Demos!

Sharing Query Results

Sharing query results is easy with the "Tags" feature. Here is a curated list of available tags.

Try Tags!

What is NeuroNLP++

NeuroNLP++ is a query interface for navigating fruit fly brain datasets. In response to free-form English queries, NeuroNLP++ links the Hemibrain dataset with the worldwide fruit fly brain literature. The linking enables the exploration of the Hemibrain dataset using cell type and general descriptors of function. It further enables in-depth investigations of the connectomic/synaptomic structure of the fly brain, using intuitive rule-based English queries.

The NeuroNLP++ query interface enables searching and locating cell types, neurons and synaptic partners with a variety of attributes of query targets (refer to the Search Bar section below).

Relevant Publications

NeuroNLP++ can be accessed from any browser supporting WebGL. Try it on your smartphone/tablet!

What Data is Accessible in NeuroNLP++.Hemibrain

    Version 1.2/1.2.1 of the Hemibrain dataset is currently available. For more information, see Louis K. Scheffer et al., A connectome and analysis of the adult Drosophila central brain. eLife 2020;9:e57443.. The dataset contains:
  • Neurons: 3D morphology, neuron name, type, data source, etc. All "segments" that are not fully traced in the hemibraindataset are also included. Their names start with "segment". However, they can be only queried using the "BodyID" assigned in the Hemibrain dataset.
  • Synapses: presynaptic and postsynaptic partners of a neuron, number of synapses and locations of synapses. Synapses of a neuron with fragments that have no morphology are bunched together.

To link the cell types of the Hemibrain dataset with the world-wide literature, we incorporated the "Drosophila anatomy and development ontologies" (see also Costa M. et al. (2013) The Drosophila anatomy ontology. Journal of Biomedical Semantics 4(32)) in the NeuroNLP++ search engine.

Want to programmatically access the data visualized here and computationally explore their functional logic?

Try FlyBrainLab, an interactive computing platform for studying the function of executable circuits constructed from fruit fly brain data.

Layout Overview

    NeuroNLP++ user interface (UI) consists of mainly 4 sections
  • Header Bar: main menu including the Get Started guide, Create/Load Tags, access to the list of visualized neurons/synapses and a list neuropils.
  • Visualization Panel: 3D visualization of queried neurons/synapses and the underlying neuropils.
  • Information Panel: detailed information including synaptic partners of neuron displayed in the visualizer.
  • Search bar: command line for natural language queries.

Search Bar

The Search Bar is the central querying interface. It supports natural language queries of neurons, synaptic partners, etc. By combining various attributes of query targets, you can create some very powerful queries. Click on the info button to the left of the search bar for more information. Also checkout the Demos!

Header Bar

The Header Bar provides access to the main menu that includes:

  • Get Started: a menu for helping users get started with NeuroNLP++.
  • Tags: a menu for creating and loading tags. Tags are 3d snapshots of stored visualization panels.
  • Neurons/Synapses: a menu displaying the visualized neurons and synapses, as well as a list of pinned neurons/synapses. Checkout the "Neurons/Synapses Menu" Demo (GUI Usage #1 in the Demos)
  • Neuropils: a menu of neuropils. Checkout the "Neuropil Menu" Demo (GUI Usage #1 in the Demos)
  • Cell Types: a list of cell types in the each of the neuropil in the Hemibrain dataset. It provides an easy way to add neurons of different cell types directly, without using the search bar.

Visualization Panel

The visualization panel provides a interactive workspace of neuron skeletons in the context of relevant neuropils. It allows you to change view, select neurons, pin neurons, etc.
To help navigating the visualized neuropils, neurons and synapses, the visualization panel provides a list of useful buttons for ease of interaction. For more details regarding the visualization panel, click on the    button on the top right corner of the panel. Also checkout the GUI Usage section of the Demos.

Information Panel

Information Panel is your major source of information regarding the individual neurons. You can view/add the pre- and post-synaptic partners of selected neuron. Check out the "Info Panel" Demo (GUI Usage #8 in the Demos).

Answers to free-form English queries will also appear in the Information Panel. You can add relevant neurons to the workspace from the buttons provided in the answer. Check out, for example, the "Free-Form Queries" Demo (NeuroNLP++ Queries #1 in the Demos).

Free-form English Queries

Launched NeuroNLP++ for Hemibrain dataset, enabling free-form English queries for cell types.

New visualization settings
    Check out the Settings menu
  • Background color can be changed.
  • More neuron rendering modes available.
  • Better balance of performance and visualization in default rendering mode.
  • Change of synapse size is reflected immediately.
Added Hemibrain "Segments"

"Segments" are fragments of neurons that have not been fully traced. Synapses to/from segments are listed in the pre/post-synaptic partners. If the morphology of a segment is available, it will appear in its own entry. All synapses from and to segments that have no morphology are grouped and listed in the entry, respectively, as segments_presynaptic_to_neuron_name and segments_postsynaptic_to_neuron_name. Segments and only segments have the word "segment" in their names. This can be used to filter them in the list.

Pre/Post-synaptic partners can be filtered by their name using regular expression

To use regular expression, start the search by /r. For example, to show only traced neurons, use /r^((?!segment).)*$ to remove all names that include "segment".

New Cell Types Menu

Cell type menu lists the names of cell types in each of the neuropil that can be added to the 3D visualization workspace. These names can be used directly in the English query without $$ any more (case insenstive). For example, "show mbon06" works, no need to use "show $MBON06$.

Improved Graph View Tool and

Now you can filter the edges in the graph by the number of synapses.

Search Bar

The Search Bar is the central querying interface. It supports free-form English queries as well as rule-based English queries of neurons, synaptic partners, etc. While you can ask any question with free-form English queries, we provide here a reference to rule-based NLP queries. By combining various attributes of query targets, you can create some very powerful queries.

    Your queries should start with a verb; the verbs supported right now are:
  • show: clear workspace and then show the queried neurons,
  • add: add to the workspace the neurons queried,
  • remove: remove from the workspace the queried neuron,
  • keep: keep in the workspace only the neurons that meet the criterion of the query,
  • hide: hide the neurons that meet the criterion of the query (this does not remove them from workspace, but reduce their visibility),
  • pin: pin the neurons that meet the criterion of the query. Pinned neurons are automatically highlighted, and cannot be removed by the "trash can" button on top of the NeuroNLP window.
  • unpin: unpin the neurons that meet the criterion of the query,
  • color: color the neurons that meet the criterion of the query with a user defined color (can be hex color code, e.g., FF0000 for red), or these predefined colors.
    Additionally, the following single word commands are supported:
  • clear: clear up the workspace, removing all neurons and synapses,
  • undo: undo your last query.
    Other short-hands:
  • show /:referenceId:[5813014882, 912147912, 880875861]: Shows the neurons whose referenceId in the original dataset is in the list. It can be used similar to \$ \\$ and regular expression and combined with other types of criteria. Examples:

Coloring: if no criteria are specified, the color will be applied to the neurons added in the most recent query. For example, if you query: show A neurons, then add B neurons, color red will color B neurons red. color A neurons 0000FF will then color A neurons blue.

To use your mouse to navigate in the 3D visualization:

Try out the "Pin/Select/Highlight" Demo (GUI Usage #3 in the Demos).

Translation

Right-click and then drag the cursor to translate the entire 3D scene.

On mobile and tablets, the scene can be translated by dragging with three fingers.

Rotation

Left-click and then drag the cursor to rotate the entire 3D scene.

On mobile and tablets, touch and drag with one finger to rotate the scene.

Zoom

Scroll up or down to zoom in or out the entire 3D scene.

On mobile and tablets, pinch out or in to zoom out or in respectively.

Highlight a Neuron/Synapse

Hover over a neuron/synapse will highlight that neuron, with all other objects dimmed (except for those pinned). Brightness of highlighted objects and dimmed objects can be respectively adjusted in the Visualization Settings (see also Visualization Settings below).

Select a Neuron/Synapse

Hover over a neuron/synapse and Left-Click to retrieve the detail information in the Info Panel.

On mobile and tablets, touch on the neuron with a single finger to select it.

Pin a Neuron/Synapse

Hover over a neuron/synapse and Double-Click to pin it in the scene. Pinned neurons will always be highlighted. Brightness of pinned objects and unpinned objects can be respectively adjusted in the Visualization Settings (see also Visualization Settings below).

After pinning neurons of interest, you can use the remove unpinned neurons icon (  ) to restrict the workspace to neurons you are interested in.

On mobile and tablets, select a neuron and then touch and hold on it for at least 750ms to pin it.

Visualization Panel Buttons

Try out the "Visualization Panel Buttons" Demo (GUI Usage #4 in the Demos).

Here we list the function of each button:

  • : Open visualization settings.
  • : Open this menu.
  • : Turn on/off stats performance monitor.
  • : Take a screenshot of the current 3D visualization. The quality of the screenshot depends on the resolution of the display and the browser window size.
  • : Reset view to original setting.
  • : Zoom in onto the visualized (not hidden) neurons.
  • : Show all neurons, synapses and neuropils.
  • : Hide all unpinned neurons/synapses.
  • : Remove all unpinned neurons/synapses.

Graph View and Connectivity Buttons

Try out the "Graph View / Connectivity Buttons" Demo (GUI Usage #5 in the Demos).

Three additional buttons in the Visualization Panel provide information about the graph of the visualized neural circuit:

  • : Open the Graph View to generate a neuronal level graph of the neurons in the current workspace. The graph is visualized in 2D. Nodes representing each neuron has the same color as in the 3D visualization. Nodes can be reordered by dragging them. Hovering over a node will highlight the corresponding neuron in the 3D visualization. Clicking on the node hide/unhide morphology of the corresponding neuron. You can adjust the synapse threshold to remove connections that has less number of synapses than the value.
  • : Open the Graph View to generate a cell type level graph of the neurons in the current workspace. Connections from all neurons of one cell type to all neurons of another cell type will be aggregated. Nodes can be reordered by dragging them. Hovering over a node will highlight the neurons of the corresponding cell type in the 3D visualization. Clicking on a node hide/unhide morphology of the neurons of the corresponding cell type.
  • : Download the graph in CSV format, that includes the pre-synaptic neuron name, post-synaptic neuron name and the number of synapses on each row.

Visualization Settings

You can customize the visual effects in the Visualization Panel by using the Visualization Settings menu. Here is a break down of the options.

Backgound
  • Color: change the color of the background of the Visualization Panel.
Neuron Rendering
  • Rendering Mode: change the way neuron morphology is rendered.
  • Default Neurite Radius: If the neuron skeleton does not contain radius information, this is the default value that will be used.
  • Neurite Radius Range: The range of acceptable radius. Any radius outside of this range will be clipped to the nearest acceptable range.
  • Default Soma Radius: If the soma of the neuron skeleton does not contain radius information, this is the default value that will be used.
  • Soma Radius Range: The range of acceptable radius of soma. Any radius outside of this range will be clipped to the nearest acceptable range.
  • Default Opacity: The opacity of visualized neurons/synapses when none of them are highlighted or pinned.
  • Higlighted Neuron Opacity: The opacity of the neuron/synapse being highlighted.
  • Low Opacity in Highlight Mode: When a neuron/synapse is highlighted, the opacity of the rest of the neurons/synapses.
  • Pinned Neuron Opacity: The opacity of pinned neurons/synapses.
  • Low Opacity in Pinned Mode: When some neurons/synapses are pinned, the opacity of the unpinned neurons/synapses.
  • Ambient Light: Brightness of ambient light onto neurons/synapses.
  • Directional Light: Brightness of directional light onto neurons/synapses.
Neuropil Rendering
  • Color: Color of the Neuropil meshes.
  • Opacity: Opacity of the Neuropil meshes.
  • Wireframe Opacity: Opacity of the wireframe on the Neuropil meshes.
  • Screen-space ambient occlusion pass (SSAO): Whether to enable SSAO effect.
  • Ambient Light: Brightness of ambient light onto neuropils.
  • Directional Light: Brightness of directional light onto neuropils.
Synapse Rendering
  • Synapse Size: Synapses are renders as a sphere with a radius specific by the value of Synapse Size.
  • Synapse Size Range: Range of valid synapse size if they are specified in the data. If the synapse size is outside of this range, it will be clipped.
  • Synapse Opacity: The opacity of the synapses when no neuron/synapse is highlighted or pinned.
PostProcessing Settings
  • Fast approXimate Anti-Aliasing (FXAA): Better anti-aliasing.
  • Bloom Pass: Create a glowing effect.