Cheetah Experimental Platform Web (CEP-Web)
Lately, we have started to work with eye trackers in order to gain a more complete understanding of the cognitive processes during the Process of Process Modeling (PPM). In this context, we came across an interesting approach for evaluating the cognitive load or mental effort of modelers using an eye tracker.
What is cognitive load and how can we measure it?
The interaction of a task at hand with the person performing the task is described by Cognitive Load Theory as cognitive load on the person solving the task. The cognitive load of a task is determined by its intrinsic load, i.e., the inherent difficulty associated with a task and its extraneous load, i.e., the load generated by the task’s representation. In our context, intrinsic load is determined by the process to be modeled. It can be characterized by the size and complexity of the model structure and constructs. In contrast, extraneous load concerns, for example, the presentation of the task to the modeler. Additionally, the properties of the notational systems, i.e., the modeling notation and the modeling environment, affect the difficulty of the modeling task, e.g., due to notational deficiencies. While the impact of various task extraneous characteristics on model understanding has been widely addressed, it is less understood how these factors impact the creation of process models. In general, research has shown that a high cognitive load increases the probability of errors, especially when the working memory’s capacity is exceeded, making cognitive load an interesting avenue for future research on the PPM. Cognitive load is typically operationalized as mental effort and various measurement techniques can be applied, such as the measurement of the diameter of the eyes’ pupil (pupillometry).
This seems interesting. Let’s measure cognitive load!
Wait a minute! Unfortunately, working with pupillometric data is not as easy as it might seem. First of all, we are talking about a large amount of data. Modern eye trackers can collect up to a 1000 data points per second. Imagine the amount of data for a modeling task lasting about 45 minutes. This can quickly result in unmanageable files that are hard to process by hand. Second, the raw data has to be processed before it can be used for analysis. For instance, we would like to identify blinks in the data stream and perform a linear interpolation on the missing data. Unfortunately, we did not come across a tool that allowed us to process the collected data as we intended. Luckily, we got a couple of people in our group who like to build software. As a result, we came up with CEP-Web! Subsequently, we will give you a glimpse of our new toy.
CEP-Web: A web platform for analyzing cognitive load using pupillometric data
When developing CEP-Web we intended to build a tool that accommodates for the specific needs of a study incorporating pupillometric data. This includes providing appropriate means for managing the data collected in the study, efficiently performing the data treatment procedure (keep in mind that processing potentially huge data files might take some time – even on machines with lots of power), and visualizing the collected data in order to assess whether the data treatment was successful. Subsequently, we will provide a brief overview regarding the different features of CEP-Web.
Study and subject management
Running one or more studies relying on pupillometric data requires more than simply exporting the data an running a program for cleaning the data. Therefore, we decided to support the full analysis cycle by also considering the management of studies and subjects in CEP-Web. The study management user interface allows you to create studies and assign subjects for each study. Whenever you upload data to CEP-Web, you will be required to assign the data to a subject. This way, we can keep track of all files associated to a subject. This becomes especially important when you would like to visualize, e.g, raw pupillometric data and cleaned pupillometric data next to each other. We will see in a minute how such a visualization might look, but first, let’s spend a minute on talking about cleaning the collected raw data.
The data treatment area is obviously the heart of CEP-Web. After uploading your files, they will appear in this area, already assigned to the corresponding subject. For each subject, you can upload an unlimited amount of files. We usually recommend to upload a video file and the corresponding pupillometric data file that can be exported from your eye tracker’s software in order to gain a comprehensive picture of the collected data. The data export is usually a tabular file with a series of columns. For running CEP-Web, at least a timestamp column, the pupil diameter for the left pupil and the diameter for the right pupil must be present (Please note that some data cleaning routines require additional data).
After uploading the data, the user can select as many pupillometric data files as desired and run so-called filters on the data. CEP-Web will take care of the rest. You can spend your time doing more important stuff! Once the filter request is completed, the user is notified and filtered files appear in the list of files. The files can then be downloaded or visualized. We are constantly working on extending the set of supported filter. Currently. CEP-Web provides the following filters that can be combined by the user in arbitrary order (Please be aware of the pre-conditions for some of the filters).
- Pupil Substitution: Replace missing data of a pupil by the data of the other pupil.
- Gaze Point Substitution: Replace missing gaze points of one eye by the gaze points of the other eye.
- Standard Deviation: Remove all pupil values that exceed three times the standard deviation
- Blink Detection: Detect blinks according to the algorithm proposed in Pedrotti, M., Lei, S., Dzaack, J., Rötting, M.: A data-driven algorithm for offline pupil signal preprocessing and eyeblink detection in low-speed eye-tracking protocols. Behavior Research Methods 43 (2011) 372-383
- Butterworth: Apply a third order lowpass Butterworth filter
- Linear Interpolation: Linear interpolate missing data
In addition, CEP-Web provides the typical data management tools for deleting, renaming, downloading and searching files. Additionally searchable tags can be added to files in order to keep track of the files. For instance, the screenshot above shows tags for raw-data, cleaned data and video files.
When implementing our data processing routine, we noted that evaluating their effect was more challenging than we expected. Comparing lengthy columns of values in order to evaluate the success of a cleaning operation turned out to be a challenging and hardly enjoyable task. Therefore, we decided to include a visualization of pupillometric data in CEP-Web.
The visualization allows you to include an unlimited number of pupillometric data lines for your subject. This screenshot includes the raw data (blue) and the corresponding cleaned data (red). As you can see, the blinks (missing data) are removed and linearly interpolated. Additionally, the filters turned the pupillopmetric data line into a smoother line, removing most of the little spikes in the raw data. In order to make the feasibility analysis easier, you might juxtapose the pupillometric data lines with the corresponding video that can be exported from the eye tracker. The video is automatically synchronized with the pupillometric data and you can directly jump to any position in the video file.
Supported eye trackers
CEP-Web was developed using the Tobii TX 300 eye tracker. The recorded data was exported using Tobii Studio to .tsv files and .avi files. Unfortunately, the .avi files exported by Tobii Studio cannot be streamed using a web browser. Therefore, the video files should be converted to a streamable video format. We made good experiences with WebM.
We would expect that the tooling should work well for Tobii eye trackers using Tobii Studio. For eye trackers of other manufacturers, we would expect a certain amount of adaptations that are required in order to deal with their data format.
How can I use CEP-Web?
Don’t worry, CEP-Web will be freely available as the software will be open source. We are currently putting the final touches on CEP-Web 1.0. Once we are done, download links and more information on how to run CEP-Web will be available here! Just keep coming back or follow us on Twitter or Facebook.