UChicago Benchmark Workflow¶

File: .github/workflows/uchicago.yml

This workflow runs automated benchmark jobs at the UChicago Analysis Facility to measure performance across different job types and configurations.

Trigger¶

The workflow runs:

• Every 6 hours (on schedule: 0 */6 * * *)
• On pull requests to main
• Manually via workflow_dispatch

Benchmark Jobs¶

The workflow runs 10 parallel benchmark jobs on arc-runner-set-uchicago runners:

Workflow Steps¶

Each job follows this pattern:

1. Checkout - Clone the repository
2. Setup Globus (if required) - Configure VOMS certificates for data access
3. Execute - Run the benchmark script
4. Parse - Parse logs and generate JSON payload (via parse action)
5. Upload to Kibana - Send results to LogStash/Kibana (via upload action)
6. Upload Logs - Store log files as GitHub artifacts

Example Job Structure¶

rucio:
  runs-on: arc-runner-set-uchicago
  steps:
    - uses: actions/checkout@v5

    - uses: ./.github/actions/setup-globus
      with:
        voms-usercert: ${{ secrets.VOMS_USERCERT }}
        voms-userkey: ${{ secrets.VOMS_USERKEY }}

    - name: execute
      run: ./Rucio/rucio_script.sh uchicago
      shell: bash
      env:
        VOMS_PASSWORD: ${{ secrets.VOMS_PASSWORD }}

    - name: parse benchmark log
      if: always()
      uses: ./.github/actions/parse
      with:
        job: ${{ github.job }}
        log-file: rucio.log
        log-type: rucio
        cluster: UC-AF
        kibana-token: ${{ secrets.KIBANA_TOKEN }}
        kibana-kind: ${{ secrets.KIBANA_KIND }}
        host: ${{ env.NODE_NAME }}
      continue-on-error: true

    - name: upload to kibana
      if: always()
      uses: ./.github/actions/upload
      with:
        payload-file: payload.json
        kibana-uri: ${{ secrets.KIBANA_URI }}
      continue-on-error: true

    - name: upload log
      if: always()
      uses: actions/upload-artifact@v4
      with:
        name: ${{ github.job }}-logs
        path: rucio.log

Monitoring Results¶

Viewing Workflow Runs¶

1. Go to the repository's Actions tab
2. Select the "uchicago" workflow
3. Click on a specific run to see job details

Checking Logs¶

• Benchmark logs: Download from workflow artifacts (available for 90 days)
• Workflow logs: View in the GitHub Actions UI
• Parsing errors: Check the "parse benchmark log" step logs
• Upload errors: Check the "upload to kibana" step logs

Common Issues¶

Benchmark job failures:

• Check VOMS credentials are valid
• Verify runner has network access to required resources
• Review benchmark script logs in artifacts

Parsing failures:

• Check that log files are generated correctly
• Verify parsing script output in "parse benchmark log" step logs
• Check token and kind values are correct

Upload failures:

• Verify payload.json was generated by parse step
• Check HTTP response status in "upload to kibana" step logs
• Verify kibana-uri is correct

Benchmark Types Explained¶

Rucio Download¶

Downloads ATLAS data files using the Rucio data management system. Measures data transfer performance.

Documentation: Rucio Download Tutorial

EVNT Generation¶

Generates Monte Carlo event files (EVNT format) using different runtime environments.

Documentation: EVNT Production Tutorial

TRUTH3 Derivation¶

Creates TRUTH3 derivation files from EVNT files for truth-level analysis.

Documentation: TRUTH3 Derivation Tutorial

NTuple to Histogram¶

Converts NTuple ROOT files to histograms using various frameworks:

• Coffea: Python-based columnar analysis framework
• FastFrames: C++ framework for fast ROOT analysis
• EventLoop: Traditional ATLAS event processing framework

Next Steps¶

• Learn about the parsing and upload action
• See the development guide for adding new benchmarks
• Check documentation workflow details