Read and Write Files

Learn how to read and write local files on the runner as part of a pipeline.

Pipelines often need to write files locally to be used in subsequent steps. For instance, we may want to write a configuration file before running a particular tool. In other cases, a previous step will have written an output file and we want to read it back into an expression to perform further processing, send data to a server, or control the pipeline flow conditions.

This tutorial demonstrates basic usage of the File step module, which is used to write local files, and the read_file() expression helper function, which is used to read local files. We will write a simple HTML file with some dynamic content. Then, we will read the file and print the content to the run output.

Step 1: Create a Pipeline

From the Pipelines page, click “New Pipeline”.

  1. Name the pipeline, and give it a description.
  2. Click “Create”. The newly created pipeline appears in the pipeline list.
  3. Click on the “Open in Builder” button.

Two step modules will be used as part of this exercise: Debug and File.

  1. Drag and drop the File step module from the left-hand sidebar into the design area.
  2. Drag and drop the Debug step module from the left-hand sidebar into the design area, just below the File step.
  3. Connect the two steps together by dragging the connector line between the two.

Step 2: Edit the Step Properties

Click on the File step. the properties sidebar will appear. There are four properties that will be edited:

  1. Step ID: This is a unique identifier for the step. It cannot contain spaces.
  2. Display Name: This is a friendly name for the step, which is displayed in the builder.
  3. File Path: The local path on the runner where the file will be written. Paths are relative to the pipeline run directory. We’re writing an HTML file in the run directory called “index.html”.
  4. Content: The file content. Template-style expressions are supported in this field. We’re using an expression to output the value of the variable called body.

Next, click on the Debug step. The properties sidebar will appear. There are 3 properties that will be edited:

  1. Step ID : This is a unique identifier for the step module. It cannot contain spaces.
  2. Display Name: This is a friendly name for the step.
  3. Message: The message to be displayed. We’ll use the read_file() helper to read the “index.html” file that was defined in the File step. The “index.html” file resides in the root directory of the agent. For more information on the read_file() helper, see the expression reference.

Step 3: Create a Variable

Next, we’ll create a variable with some example content for the HTML body.

In the upper right corner of the builder, click “Variables”, then click “Add Variable”.

The “Edit Variable” window appears. The variable consists of 3 components:

  1. Variable Type: Data type of the variable.
  2. Name: A friendly name for the variable.
  3. Key: The variable key. This key should match the vars.body expression we used previously.
  4. Value: The variable value. This value will be printed into the HTML body of the file we’re writing.

Click “Submit” to save the HTML Body variable.

To save the pipeline, click “Save”. The “Save Pipeline” dialog box will appear. Enter any desired notes about the pipeline and click “Save & Exit”.

Step 4: Run the Pipeline

Now we’ll run our newly created pipeline. Click the Run button in the top-left.

The Run Pipeline dialog box appears. Click the Run button at the bottom.

You should now see the run output. If successful, you should be able to see the final HTML content that was written to the file.