Skip to Main Content

Data Farm Quick-Start: Query data

Query data

Once you have data access to a dataset, you can further explore the dataset by adding it to a project. The Redivis project interface allows you to develop scalable, reproducible, and self-documenting data pipelines as you filter, transform, and merge data from across Redivis.

To begin, select the Add to project button. From the drop-down, you can add the dataset to an existing project, or create a new project.

screenshot of selecting Add to Project button, as explained on this page

screenshot of adding a dataset to an existing or new project, as explained on this page

This will bring the dataset into the Redivis project interface. Select the dataset icon to view the tables in the dataset. You can select a table to view its variables and summary statistics. In this case, we will select the articles_1977 table.

screenshot of selecting a dataset in the visual query editor and viewing its tables, as explained on this page

screenshot of selecting a table in the visual query editor and viewing its variables, as explained on this page

To manipulate the data, click the Transform node underneath the table icon. This will open a panel on the right side of the page where you can 1) apply certain transformations, and 2) select variables to include in the output.

screenshot of selecting the transform node, as explained on this page

screenshot of transform node panel, as explained on this page

In this example, I want to filter the data to only include articles from the Business section. For my future reference, I rename the transform "Filter by Business section" and add a Filter step.

screenshot of renaming the transform "Filter by Business section" and selecting a Filter step, as explained on this page

➎ I specify the rules for my Filter step (section LIKE 'business'), and select four variables for inclusion in my output: section, title, authors and paragraphs. Then I click Run.

screenshot of  setting the transform rules, specifically 1) defining the Filter and 2) selecting variables to include in the output, as explained on this page

screenshot of selecting Run button, as explained on this page

My output appears as a new table in the visual query editor. I can also see more information about my output on the right side of the page.

screenshot of the transform output in the visual query editor, as explained on this page. The transform output is a new table.