NeuroNLP++ is a query interface for navigating fruit fly brain datasets. In response to free-form English queries, NeuroNLP++ links the Hemibrain dataset with the worldwide fruit fly brain literature. The linking enables the exploration of the Hemibrain dataset using cell type and general descriptors of function. It further enables in-depth investigations of the connectomic/synaptomic structure of the fly brain, using intuitive rule-based English queries. The NeuroNLP++ query interface enables searching and locating cell types, neurons and synaptic partners with a variety of attributes of query targets (refer to the Search Bar section below). Relevant Publications For rule-based English queries: see Nikul H. Ukani et al., The Fruit Fly Brain Observatory: From Structure to Function, bioRxiv 2019, 580290, March 2019. For free-form English queries: see Aurel A. Lazar, et al., A Programmable Ontology Encompassing the Functional Logic of the Drosophila Brain, Frontiers in Neuroinformatics, June 2022. NeuroNLP++ can be accessed from any browser supporting WebGL. Try it on your smartphone/tablet!
Version 1.2/1.2.1 of the Hemibrain dataset is currently available. For more information, see Louis K. Scheffer et al., A connectome and analysis of the adult Drosophila central brain. eLife 2020;9:e57443.. The dataset contains: Neurons: 3D morphology, neuron name, type, data source, etc. All "segments" that are not fully traced in the hemibraindataset are also included. Their names start with "segment". However, they can be only queried using the "BodyID" assigned in the Hemibrain dataset. Synapses: presynaptic and postsynaptic partners of a neuron, number of synapses and locations of synapses. Synapses of a neuron with fragments that have no morphology are bunched together. To link the cell types of the Hemibrain dataset with the world-wide literature, we incorporated the "Drosophila anatomy and development ontologies" (see also Costa M. et al. (2013) The Drosophila anatomy ontology. Journal of Biomedical Semantics 4(32)) in the NeuroNLP++ search engine. Want to programmatically access the data visualized here and computationally explore their functional logic? Try FlyBrainLab, an interactive computing platform for studying the function of executable circuits constructed from fruit fly brain data.
NeuroNLP++ user interface (UI) consists of mainly 4 sections Header Bar: main menu including the Get Started guide, Create/Load Tags, access to the list of visualized neurons/synapses and a list neuropils. Visualization Panel: 3D visualization of queried neurons/synapses and the underlying neuropils. Information Panel: detailed information including synaptic partners of neuron displayed in the visualizer. Search bar: command line for natural language queries.
The Search Bar is the central querying interface. It supports natural language queries of neurons, synaptic partners, etc. By combining various attributes of query targets, you can create some very powerful queries. Click on the info button to the left of the search bar for more information. Also checkout the Demos!
The Header Bar provides access to the main menu that includes: Get Started: a menu for helping users get started with NeuroNLP++. Tags: a menu for creating and loading tags. Tags are 3d snapshots of stored visualization panels. Neurons/Synapses: a menu displaying the visualized neurons and synapses, as well as a list of pinned neurons/synapses. Checkout the "Neurons/Synapses Menu" Demo (GUI Usage #1 in the Demos) Neuropils: a menu of neuropils. Checkout the "Neuropil Menu" Demo (GUI Usage #1 in the Demos) Cell Types: a list of cell types in the each of the neuropil in the Hemibrain dataset. It provides an easy way to add neurons of different cell types directly, without using the search bar.
The visualization panel provides a interactive workspace of neuron skeletons in the context of relevant neuropils. It allows you to change view, select neurons, pin neurons, etc. To help navigating the visualized neuropils, neurons and synapses, the visualization panel provides a list of useful buttons for ease of interaction. For more details regarding the visualization panel, click on the button on the top right corner of the panel. Also checkout the GUI Usage section of the Demos.
Information Panel is your major source of information regarding the individual neurons. You can view/add the pre- and post-synaptic partners of selected neuron. Check out the "Info Panel" Demo (GUI Usage #8 in the Demos). Answers to free-form English queries will also appear in the Information Panel. You can add relevant neurons to the workspace from the buttons provided in the answer. Check out, for example, the "Free-Form Queries" Demo (NeuroNLP++ Queries #1 in the Demos).
The Search Bar is the central querying interface. It supports free-form English queries as well as rule-based English queries of neurons, synaptic partners, etc. While you can ask any question with free-form English queries, we provide here a reference to rule-based NLP queries. By combining various attributes of query targets, you can create some very powerful queries. Your queries should start with a verb; the verbs supported right now are: show: clear workspace and then show the queried neurons, add: add to the workspace the neurons queried, remove: remove from the workspace the queried neuron, keep: keep in the workspace only the neurons that meet the criterion of the query, hide: hide the neurons that meet the criterion of the query (this does not remove them from workspace, but reduce their visibility), pin: pin the neurons that meet the criterion of the query. Pinned neurons are automatically highlighted, and cannot be removed by the "trash can" button on top of the NeuroNLP window. unpin: unpin the neurons that meet the criterion of the query, color: color the neurons that meet the criterion of the query with a user defined color (can be hex color code, e.g., FF0000 for red), or these predefined colors. Additionally, the following single word commands are supported: clear: clear up the workspace, removing all neurons and synapses, undo: undo your last query. Next, we explain the rules for defining the criterion of the query, using the verb show as an example. show cell-type neurons: Shows the neurons of the cell type. Example: show PEG neurons, or simply show peg. show $string$ neurons: Shows neurons with a name that contains the string. Example: show $PEG$ neurons, or simply show $PEG$, will query any neuron whose name contain the string *PEG*. show /rstring/r neurons: Show neurons whose name matches the regular expressing string (This requires some knowledge of how the neurons are named in each dataset). Example: show /r(.*)PEG(.*)R1(.*)/r neurons, or simply show /r(.*)PEG(.*)R1(.*)/r will show the PEG neurons that innervate PB glomerulus R1. show neurons in|that innervate|that arborize in neuropil/subregion: Shows neurons that has output or input in a neuropil or a subregion of a neuropil. Examples: show neurons in ellipsoid body, or using abbreviations show neurons in EB, show PEG neurons that arborize in PB glomerulus r9. show neurons that innervate right al and right mb. Note that this is different from show neurons that innervate right al or right mb. show local neurons in neuropil: Shows the neurons that has only inputs and outputs within the neuropil (note that due to lack of data in some datasets, some neurons are only traced in one neuropil and thus are classified local neuron by default). Examples: show local neurons in ellipsoid body show neurons with|that have inputs|outputs in neuropil/subregion: More specific then the previous query on the inputs or outputs. Examples: show neurons with inputs in AME, show ER3w neurons that have outputs in EB, show neurons with inputs in right antennal lobe and outputs in right lateral horn, or equivalently show neurons projecting from right antennal lobe to right lateral horn, show neurons that connect right AL and right MB. Includes both the neurons that has inputs in AL and outputs in MB, and those has inputs in MB and outputs in AL. show neurons presynaptic|postsynaptic to: Shows the neurons that are presynaptic or postsynaptic to the neurons defined after the word to. Exmples: show neurons presynaptic to TuBu05, show $aMe$ presynaptic to KCs that innervate alpha'1 compartment, show DAN postsynaptic to MBONs with at least 30 synapses. show presynaptic|postsynaptic neurons: Shows the neurons that are presynaptic or postsynaptic to the neurons already in workspace. Examples: show presynaptic neurons, show postsynaptic neurons with at least 10 synapses, show postynaptic MBON that innervate gamma lobe with at least 5 synapses. This searches for MBONs that satisfy: 1) postsyantpic to the neurons in the workspace with at least 5 synapses and 2) innervate gamma lobe of the mushroom body. Other short-hands: show /:referenceId:[5813014882, 912147912, 880875861]: Shows the neurons whose referenceId in the original dataset is in the list. It can be used similar to \$ \\$ and regular expression and combined with other types of criteria. Examples: show /:referenceId:[5813014882, 912147912, 880875861] Coloring: if no criteria are specified, the color will be applied to the neurons added in the most recent query. For example, if you query: show A neurons, then add B neurons, color red will color B neurons red. color A neurons 0000FF will then color A neurons blue.
To use your mouse to navigate in the 3D visualization: Try out the "Pin/Select/Highlight" Demo (GUI Usage #3 in the Demos). Translation Right-click and then drag the cursor to translate the entire 3D scene. On mobile and tablets, the scene can be translated by dragging with three fingers. Rotation Left-click and then drag the cursor to rotate the entire 3D scene. On mobile and tablets, touch and drag with one finger to rotate the scene. Zoom Scroll up or down to zoom in or out the entire 3D scene. On mobile and tablets, pinch out or in to zoom out or in respectively. Highlight a Neuron/Synapse Hover over a neuron/synapse will highlight that neuron, with all other objects dimmed (except for those pinned). Brightness of highlighted objects and dimmed objects can be respectively adjusted in the Visualization Settings (see also Visualization Settings below). Select a Neuron/Synapse Hover over a neuron/synapse and Left-Click to retrieve the detail information in the Info Panel. On mobile and tablets, touch on the neuron with a single finger to select it. Pin a Neuron/Synapse Hover over a neuron/synapse and Double-Click to pin it in the scene. Pinned neurons will always be highlighted. Brightness of pinned objects and unpinned objects can be respectively adjusted in the Visualization Settings (see also Visualization Settings below). After pinning neurons of interest, you can use the remove unpinned neurons icon ( ) to restrict the workspace to neurons you are interested in. On mobile and tablets, select a neuron and then touch and hold on it for at least 750ms to pin it.
Try out the "Visualization Panel Buttons" Demo (GUI Usage #4 in the Demos). Here we list the function of each button: : Remove all unpinned neurons/synapses. : Hide all unpinned neurons/synapses. : Show all neurons, synapses and neuropils. : Zoom in onto the visualized (not hidden) neurons. : Reset view to original setting. : Open this menu. : Take a screenshot of the current 3D visualization. The quality of the screenshot depends on the resolution of the display and the browser window size.
Try out the "Graph View / Connectivity Buttons" Demo (GUI Usage #5 in the Demos). Three additional buttons in the Visualization Panel provide information about the graph of the visualized neural circuit: : Open the Graph View to generate a neuronal level graph of the neurons in the current workspace. The graph is visualized in 2D. Nodes representing each neuron has the same color as in the 3D visualization. Nodes can be reordered by dragging them. Hovering over a node will highlight the corresponding neuron in the 3D visualization. Clicking on the node hide/unhide morphology of the corresponding neuron. You can adjust the synapse threshold to remove connections that has less number of synapses than the value. : Open the Graph View to generate a cell type level graph of the neurons in the current workspace. Connections from all neurons of one cell type to all neurons of another cell type will be aggregated. Nodes can be reordered by dragging them. Hovering over a node will highlight the neurons of the corresponding cell type in the 3D visualization. Clicking on a node hide/unhide morphology of the neurons of the corresponding cell type. : Download the graph in CSV format, that includes the pre-synaptic neuron name, post-synaptic neuron name and the number of synapses on each row.
You can customize the visual effects in the Visualization Panel by using the Visualization Settings menu. Here is a break down of the options. Backgound Color: change the color of the background of the Visualization Panel. Neuron Rendering Rendering Mode: change the way neuron morphology is rendered. Default Neurite Radius: If the neuron skeleton does not contain radius information, this is the default value that will be used. Neurite Radius Range: The range of acceptable radius. Any radius outside of this range will be clipped to the nearest acceptable range. Default Soma Radius: If the soma of the neuron skeleton does not contain radius information, this is the default value that will be used. Soma Radius Range: The range of acceptable radius of soma. Any radius outside of this range will be clipped to the nearest acceptable range. Default Opacity: The opacity of visualized neurons/synapses when none of them are highlighted or pinned. Higlighted Neuron Opacity: The opacity of the neuron/synapse being highlighted. Low Opacity in Highlight Mode: When a neuron/synapse is highlighted, the opacity of the rest of the neurons/synapses. Pinned Neuron Opacity: The opacity of pinned neurons/synapses. Low Opacity in Pinned Mode: When some neurons/synapses are pinned, the opacity of the unpinned neurons/synapses. Ambient Light: Brightness of ambient light onto neurons/synapses. Directional Light: Brightness of directional light onto neurons/synapses. Neuropil Rendering Color: Color of the Neuropil meshes. Opacity: Opacity of the Neuropil meshes. Wireframe Opacity: Opacity of the wireframe on the Neuropil meshes. Screen-space ambient occlusion pass (SSAO): Whether to enable SSAO effect. Ambient Light: Brightness of ambient light onto neuropils. Directional Light: Brightness of directional light onto neuropils. Synapse Rendering Synapse Size: Synapses are renders as a sphere with a radius specific by the value of Synapse Size. Synapse Size Range: Range of valid synapse size if they are specified in the data. If the synapse size is outside of this range, it will be clipped. Synapse Opacity: The opacity of the synapses when no neuron/synapse is highlighted or pinned. PostProcessing Settings Fast approXimate Anti-Aliasing (FXAA): Better anti-aliasing. Bloom Pass: Create a glowing effect.