What is open data?

Blender Open Data is a platform to collect, display and query the results of hardware and software performance tests - provided by the Blender community. Using this benchmark we aim to compare performance variation in software and hardware to support the Blender development process.

As initial guideline for our definition of Blender Open Data, we were inspired by the Eight Principles of Open Data.

We believe that Blender Open Data should be:

  • Complete - All public data is made available. Public data is data that is not subject to valid privacy, security or privilege limitations.
  • Primary - Data is as collected at the source, with the highest possible level of granularity, not in aggregate or modified forms.
  • Timely - Data is made available as quickly as necessary to preserve the value of the data.
  • Accessible - Data is available to the widest range of users for the widest range of purposes.
  • Machine processable - Data is reasonably structured to allow automated processing.
  • Non-discriminatory - Data is available to anyone, with no requirement of registration.
  • Non-proprietary - Data is available in a format over which no entity has exclusive control.
  • License-free - Data is not subject to any copyright, patent, trademark or trade secret regulation. Reasonable privacy, security and privilege restrictions may be allowed.
  • Online and Free - Information is not meaningfully public if it is not available on the Internet at no charge, or at least no more than the marginal cost of reproduction. It should also be findable.
  • Permanent - Data should be made available at a stable Internet location indefinitely and in a stable data format for as long as possible.

We take a privacy-conscious approach when handling Benchmark data.

How does it work?

Benchmark

You can download the Open Data Benchmark from the opendata.blender.org homepage with versions for Windows, Linux and macOS. You can then select any number of the 7 benchmarks (details listed below) to run on your choice of Blender version and render device (CPU / GPU). The benchmark will also gather non-identifiable data on your system setup, details of which can be found below. Once the benchmark is complete you can share your results on Blender Open Data publicly.

Anonymous Data

All data is kept anonymous by default. No personally identifiable information is collected or displayed. You can however enable a display name to be shown with your results in the Open Data settings.

Run Offline

You can also run the benchmark in a completely offline environment. In order to do so, download and extract the benchmark script, download and extract the scene you are interested in, enter the script directory and run the following command: 

path-to-blender-executable --background \
  --factory-startup \
  -noaudio \
  --debug-cycles \
  --enable-autoexec \
  --engine \
  CYCLES \
  scene-path.blend \
  --python \
  main.py \
  -- \
  --device-type CPU

This will output the benchmark result as the launcher would.

Benchmark Score

The Blender benchmark Score is a measure of how quickly Cycles can render path tracing samples on one CPU or GPU device. The higher the number, the better. In particular it's the estimated number of samples per minute, summed for all benchmark scenes.

Uninstalling Benchmark Launcher

First, remove the Blender Benchmark launcher from your system. Then, remove the benchmark scenes and Blender binaries downloaded by the launcher. Depending on the operating system, they are located at:

  • linux: ~/.cache/blender-benchmark-launcher
  • macOS: ~/Library/Caches/blender-benchmark-launcher
  • windows: C:\Users\<_username_>\AppData\Local\<_AppAuthor_>\<_AppName_>\Cache

What data do we collect?

When running a benchmark, the software will collect some non-identifiable information about your system for analysis. For example, we collect the operating system (E.g. Windows, Linux or macOS) to compare how efficient Blender runs on each OS.

You can expand the sample below which shows an example of all the data we collect, additionally you can find the raw data for your submissions in each benchmark's detail view. 


      

    

  

  

    

      

        
Accessing the data

        
The opendata.blender.org website is designed to be an easy way to view, query and access the benchmark data.
You can use the Search Data page to query, sort and download the data
(in JSON and CSV format). You can also view the daily snapshot
of the data in its raw JSON format. All of the past snapshots are available at opendata.blender.org/snapshots/.
        

      

    

  


  

    

      

        
Contact

        
If you would like to get in touch with a suggestion, write to francesco@blender.org.
Keep in mind that for technical support is always best to use the issue tracker on developer.blender.org.
        

      

    

  

  

    

      

        
Credits

        
The project has been developed by the team at Blender: Brecht van Lommel, Dan MacGrath,
Francesco Siddi, Pablo Vazquez, Sybren Stüvel, Sergey Sharybin, Sem Mulder and Mike Newbon.
This project was commissioned by Ton Roosendaal, chairman Blender Foundation.