52 of 128 fail task construction: staged or simulated demonstrations,
recording attributes or source identities used as labels, or tasks redefined by the archive.
41 have a leakage concern: 17 contain test series identical to training series;
24 more share a source across the split. 65 splits are Unclear, not verified clean.
Only 38 are real tasks on series sampled over time; 10 of
them have a leakage concern. Real tasks include cases with undocumented label meaning.
How to read the verdicts
Task
Fails for toy demonstrations, or when the archive built the label instead of using a real source task.
Metadata as label: a recording attribute, such as season or device.
Instance identity as label: who or what produced the series.
Task replacement: a different task defined by the archive.
Toy source task: data staged or simulated to demonstrate a method.
Undocumented label meaning is counted with real tasks.
Split
Concern: the test set shares series or a source with training. Unclear: not enough
evidence to check; not a verified clean split. Separated: training and test come from different sources, as documented or by construction when each series is a distinct event or object with no known grouping above that unit. Related periods: different windows of
the same ongoing process. Generated: source separation is not assessed for generated data. Verdicts depend on
the assumed new case shown in each record.
Data origin
Temporal: sampled over real or simulated time. Ordered, non-temporal: indexed by wavelength, contour,
frequency or temperature, or time axes joined end to end. Synthetic: indexed by a generator step.
Evidence
S source textD released dataR inferenceI inference about use
Fails task construction and leakage 24
Fails task construction 28
Fails leakage 17
Real task, leakage unknown 53
Real task, no leakage found 6
Task
Split
Data origin
More filters
Why tasks fail
Leakage evidence
Overall
Domain
Multichannel source
Dataset family
Source task
Label origin
Label attached by
Series index
DatasetTaskSplitData originNotes
ACSF1DEVICE · Energy & appliances · 100/100 train/test · length 1460 · 10 classesreal tasksource task labelUnclearassumed new case: new appliancestemporaltimesingle channelopen question
Taskreal task
Source task
real task appliance category (10 classes of household device)
Archive task
appliance category (10 classes of household device)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New appliances
Exact duplicates
none found D
Why
The 100/100 split matches ACS-F1's Test Protocol 1.0, which trains on the first acquisition session and tests on the second session of the same appliances; the archive does not say which protocol was used, and the series length of 1460 does not match the documented one-hour sessions. Whether the test presents a further session of familiar appliances, or appliances the model has never seen, therefore cannot be established. A concern would arise if the result were presented as performance on new appliances.
Source
ACS-F1 is a database of appliance consumption signatures built at the iCoSys institute (HES-SO Fribourg) explicitly to support appliance recognition: the project's own page states the data are "appliance consumption signatures and categories to be used for appliance recognition tasks".
Publication
Gisler, Ridi, Zufferey, Khaled, Hennebert, "Appliance consumption signature database and recognition test protocols", WoSSPA 2013 (the primary paper was not read -- IEEE Xplore is paywalled). The database description page was read: https://icosys.ch/acs-f1 -- "2 sessions of 1 hour each", "Acquisition sampling frequency: 10^-1 Hz", "about 100 home appliances divided into 10 categories". The UCR version was extracted by Schaefer & Leser (WEASEL, CIKM 2017).
Full record
Benchmark task
Classify a 1460-point single-appliance consumption trace into one of 10 appliance categories, with 100 train and 100 test series.
Who would use it
Intrusive/plug-level load monitoring and smart-plug or smart-home products that must recognise what is plugged in; also a reference corpus for NILM research.
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (appliance category (10 classes of household device)); label origin: source task.
Data origin
temporalindex: time
Archive changes
z-normalization
Leakage
unknownsource separation undocumented
Split sources
100/100 matches the ACS-F1 Test Protocol 1.0 (train on acquisition session 1, test on session 2 of the same appliances); the archive page does not state which protocol was used RR
Multichannel source
yes, one channel from a multichannel recording
ACS-F1 recorded six electrical quantities per appliance (real and reactive power, RMS current, frequency, RMS voltage, phase); the UCR series keeps one (power consumption), dropping reactive power and phase.
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New appliances (units not metered before).
Split evidence
the 100/100 sizes match ACS-F1 Test Protocol 1.0, which trains on session 1 and tests on session 2 of the same appliance instances; the archive page does not state the protocol and the series length does not reconcile with the documented sessions, so the split rule is unverified R; a plug-level recogniser is deployed on appliances it has not metered before I.
Open questions
If the split follows Test Protocol 1.0 it separates recording sessions of the same appliance instances, not appliances; that reading is unverified.
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New independently collected samples
Exact duplicates
none found D
Why
The cases are outlines of tiny biological specimens. We do not know whether samples grown from the same biological source, or photographed on the same microscope slide, occur in both sets. They could share growth or imaging conditions. The records do not let us check whether the test samples were collected independently.
Source
The ADIAC project (Automatic Diatom Identification And Classification, EU MAST programme, 1998-2001) set out to automate identification of diatoms (unicellular algae) from light-microscope images.
Publication
Jalba, Wilkinson & Roerdink, "Automatic Segmentation of Diatom Images for Classification", Microscopy Research and Technique 65:72-85, 2004 (read via the Groningen repository copy) - Experimental Results: "All segmentation methods described in the previous section were applied to a database of 808 diatom images comprising 37 different diatom taxa, each taxon (class in the pattern recognition sense) having at least 20 representatives". ADIAC project site (RBGE): "The overall aim of ADIAC was to develop appropriate image databases and analytical methods for the automated identification of diatoms for use in environmental monitoring, micropalaeontology and forensic research etc."
Full record
Benchmark task
Classify a univariate length-176 contour series into one of 37 diatom taxa.
Who would use it
water-quality and ecological monitoring labs, diatom taxonomists, micropalaeontologists and forensic scientists
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (diatom taxon (37 taxa)); label origin: source task.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New independently collected samples.
Split evidence
one outline per diatom valve; nothing documents whether valves of the same clone or slide sit on both sides R.
AllGestureWiimoteXHAR · Human motion & gesture · 300/700 train/test · length variable · 10 classesfailstask replacementUnclearassumed new case: new peopletemporaltimesingle channelfamily: wiimote-gesturesopen question
Taskfails
Source task
real task User identity; gesture was a protocol factor
Archive task
Which of ten gestures
Label attached by
CuratorsredefinedSR
Verdict
failstask replacement
The source study posed its own prediction task, which the archive did not keep as the label.
SplitUnclear
New case assumed
New people
Exact duplicates
none found D
Why
The data contain gestures from ten people. The archive shuffled and sampled recordings, but does not say that all recordings from a person were kept in one set. We therefore do not know whether test users are new. This matters if the intended application is to recognise gestures from people the model has never seen.
Source
The recordings come from a biometrics study, not a gesture-vocabulary study: Guna, Humar and Pogacnik built an implicit user-identification system in which a person is recognised from the way they pick up or handle a Wiimote, aimed at low-stakes login (e.g. multimedia services).
Publication
Guna, Humar, Pogacnik, "Intuitive gesture based user identification system", 35th Int. Conf. on Telecommunications and Signal Processing (TSP), IEEE 2012. Full text not read - the paper is closed access (OpenAlex reports oa_status "closed", no repository copy). Its abstract was read in full via the OpenAlex API: "We present an intuitive and very easy to use implicit gesture based identification system that is especially suited for security-wise non-critical applications, such as the user login in the multimedia services. The term 'implicit gesture' in our work refers to a natural physical hand manipulation performed by the user who takes hold of the control device and simply picks it up from its neutral motionless position - a 'pick-up' gesture... 100% recognition accuracy was achieved when using explicit identification signature gestures and over 91% recognition accuracy was achieved when using the implicit 'pick-up' gesture."
Full record
Benchmark task
Classify a variable-length single-axis Wiimote acceleration series into one of ten gesture classes.
Who would use it
gesture-driven interfaces for consumer devices; the source study's own use case was implicit user identification for logging in to multimedia services
Task construction
failtask replacement
Benchmark target: Which of ten gestures. Source target or purpose: User identity; gesture was a protocol factor. Label attached by: Curators. Mechanism 'task replacement': the archive replaces the source's label with a different task, or defines classes by its own rule where the source posed none.
Data origin
temporalindex: time
Archive changes
channel-selection
Leakage
unknownsource separation undocumented
Split sources
Archive page: instances 'shuffled and randomly sampled so that instances across datasets are not synchronized by dimension or subject'; 300 train / 700 test; no subject ids ship SIR
Multichannel source
yes, one channel from a multichannel recording
The source recorded three-axis acceleration from a hand-held Nintendo Wii remote, one recording per gesture performance; the UCR release keeps a single channel of it. Two of three accelerometer axes are dropped, and because the instances were shuffled and subsampled independently per axis, the multivariate recording cannot be reconstructed. Subject identity - the source study's actual label - is not released. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
AllGestureWiimoteX/Y/Z are the three acceleration axes of the same recordings; PickupGestureWiimoteZ and ShakeGestureWiimoteZ are the z axis of two gestures from the same 10 subjects, labelled by user
Overall verdict
fails task construction (leakage unknown)
New case assumed
New people.
Split evidence
the archive's 'shuffled and randomly sampled' sentence states its own purpose as desynchronising the X/Y/Z datasets ('so that instances across datasets are not sychronized by dimension or subject'), not as a description of the train/test policy; no subject id is released, and the 300/700 split is exactly 3x100 / 7x100 of the 10 subjects x 10 gestures x 10 repetitions, so a subject-disjoint split is numerically consistent with the released sizes R. Subject sharing is therefore not established; a gesture interface for consumer devices is used by people it was not trained on I.
Label
archive page: 'Classes are based on gestures', with the gesture names given as the 'original label' (donated by J. Guna); the gesture classes are the collectors' protocol factor, but the gesture-recognition task was posed by the archive, not by the collectors, whose target was user identity R.
Open questions
Subject sharing inferred from the source design (R) but not documented, so Unclear rather than Concern.
AllGestureWiimoteYHAR · Human motion & gesture · 300/700 train/test · length variable · 10 classesfailstask replacementUnclearassumed new case: new peopletemporaltimesingle channelfamily: wiimote-gesturesopen question
Taskfails
Source task
real task User identity; gesture was a protocol factor
Archive task
Which of ten gestures
Label attached by
CuratorsredefinedSR
Verdict
failstask replacement
The source study posed its own prediction task, which the archive did not keep as the label.
SplitUnclear
New case assumed
New people
Exact duplicates
none found D
Why
Same as AllGestureWiimoteX. [AllGestureWiimoteX: The data contain gestures from ten people. The archive shuffled and sampled recordings, but does not say that all recordings from a person were kept in one set. We therefore do not know whether test users are new. This matters if the intended application is to recognise gestures from people the model has never seen.]
Source
The recordings come from a biometrics study, not a gesture-vocabulary study: Guna, Humar and Pogacnik built an implicit user-identification system in which a person is recognised from the way they pick up or handle a Wiimote, aimed at low-stakes login (e.g. multimedia services).
Publication
Guna, Humar, Pogacnik, "Intuitive gesture based user identification system", 35th Int. Conf. on Telecommunications and Signal Processing (TSP), IEEE 2012. Full text not read - the paper is closed access (OpenAlex reports oa_status "closed", no repository copy). Its abstract was read in full via the OpenAlex API: "We present an intuitive and very easy to use implicit gesture based identification system that is especially suited for security-wise non-critical applications, such as the user login in the multimedia services. The term 'implicit gesture' in our work refers to a natural physical hand manipulation performed by the user who takes hold of the control device and simply picks it up from its neutral motionless position - a 'pick-up' gesture... 100% recognition accuracy was achieved when using explicit identification signature gestures and over 91% recognition accuracy was achieved when using the implicit 'pick-up' gesture."
Full record
Benchmark task
Classify a variable-length single-axis Wiimote acceleration series into one of ten gesture classes.
Who would use it
gesture-driven interfaces for consumer devices; the source study's own use case was implicit user identification for logging in to multimedia services
Task construction
failtask replacement
Benchmark target: Which of ten gestures. Source target or purpose: User identity; gesture was a protocol factor. Label attached by: Curators. Mechanism 'task replacement': the archive replaces the source's label with a different task, or defines classes by its own rule where the source posed none.
Data origin
temporalindex: time
Archive changes
channel-selection
Leakage
unknownsource separation undocumented
Split sources
Archive page: instances 'shuffled and randomly sampled so that instances across datasets are not synchronized by dimension or subject'; 300 train / 700 test; no subject ids ship SIR
Multichannel source
yes, one channel from a multichannel recording
The source recorded three-axis acceleration from a hand-held Nintendo Wii remote, one recording per gesture performance; the UCR release keeps a single channel of it. Two of three accelerometer axes are dropped, and because the instances were shuffled and subsampled independently per axis, the multivariate recording cannot be reconstructed. Subject identity - the source study's actual label - is not released. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
AllGestureWiimoteX/Y/Z are the three acceleration axes of the same recordings; PickupGestureWiimoteZ and ShakeGestureWiimoteZ are the z axis of two gestures from the same 10 subjects, labelled by user
Overall verdict
fails task construction (leakage unknown)
New case assumed
New people.
Split evidence
the archive's 'shuffled and randomly sampled' sentence states its own purpose as desynchronising the X/Y/Z datasets ('so that instances across datasets are not sychronized by dimension or subject'), not as a description of the train/test policy; no subject id is released, and the 300/700 split is exactly 3x100 / 7x100 of the 10 subjects x 10 gestures x 10 repetitions, so a subject-disjoint split is numerically consistent with the released sizes R. Subject sharing is therefore not established; a gesture interface for consumer devices is used by people it was not trained on I.
Label
as AllGestureWiimoteX. The gesture classes are the collectors' protocol factor, but the gesture-recognition task was posed by the archive editor; the collectors' target was user identity R. Subject sharing across the split is not established (no ids; the sizes are consistent with a subject-disjoint split) R.
Open questions
Subject sharing inferred from the source design (R) but not documented, so Unclear rather than Concern.
AllGestureWiimoteZHAR · Human motion & gesture · 300/700 train/test · length variable · 10 classesfailstask replacementUnclearassumed new case: new peopletemporaltimesingle channelfamily: wiimote-gesturesopen question
Taskfails
Source task
real task User identity; gesture was a protocol factor
Archive task
Which of ten gestures
Label attached by
CuratorsredefinedSR
Verdict
failstask replacement
The source study posed its own prediction task, which the archive did not keep as the label.
SplitUnclear
New case assumed
New people
Exact duplicates
none found D
Why
Same as AllGestureWiimoteX. [AllGestureWiimoteX: The data contain gestures from ten people. The archive shuffled and sampled recordings, but does not say that all recordings from a person were kept in one set. We therefore do not know whether test users are new. This matters if the intended application is to recognise gestures from people the model has never seen.]
Source
The recordings come from a biometrics study, not a gesture-vocabulary study: Guna, Humar and Pogacnik built an implicit user-identification system in which a person is recognised from the way they pick up or handle a Wiimote, aimed at low-stakes login (e.g. multimedia services).
Publication
Guna, Humar, Pogacnik, "Intuitive gesture based user identification system", 35th Int. Conf. on Telecommunications and Signal Processing (TSP), IEEE 2012. Full text not read - the paper is closed access (OpenAlex reports oa_status "closed", no repository copy). Its abstract was read in full via the OpenAlex API: "We present an intuitive and very easy to use implicit gesture based identification system that is especially suited for security-wise non-critical applications, such as the user login in the multimedia services. The term 'implicit gesture' in our work refers to a natural physical hand manipulation performed by the user who takes hold of the control device and simply picks it up from its neutral motionless position - a 'pick-up' gesture... 100% recognition accuracy was achieved when using explicit identification signature gestures and over 91% recognition accuracy was achieved when using the implicit 'pick-up' gesture."
Full record
Benchmark task
Classify a variable-length single-axis Wiimote acceleration series into one of ten gesture classes.
Who would use it
gesture-driven interfaces for consumer devices; the source study's own use case was implicit user identification for logging in to multimedia services
Task construction
failtask replacement
Benchmark target: Which of ten gestures. Source target or purpose: User identity; gesture was a protocol factor. Label attached by: Curators. Mechanism 'task replacement': the archive replaces the source's label with a different task, or defines classes by its own rule where the source posed none.
Data origin
temporalindex: time
Archive changes
channel-selection
Leakage
unknownsource separation undocumented
Split sources
Archive page: instances 'shuffled and randomly sampled so that instances across datasets are not synchronized by dimension or subject'; 300 train / 700 test; no subject ids ship SIR
Multichannel source
yes, one channel from a multichannel recording
The source recorded three-axis acceleration from a hand-held Nintendo Wii remote, one recording per gesture performance; the UCR release keeps a single channel of it. Two of three accelerometer axes are dropped, and because the instances were shuffled and subsampled independently per axis, the multivariate recording cannot be reconstructed. Subject identity - the source study's actual label - is not released. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
AllGestureWiimoteX/Y/Z are the three acceleration axes of the same recordings; PickupGestureWiimoteZ and ShakeGestureWiimoteZ are the z axis of two gestures from the same 10 subjects, labelled by user
Overall verdict
fails task construction (leakage unknown)
New case assumed
New people.
Split evidence
the archive's 'shuffled and randomly sampled' sentence states its own purpose as desynchronising the X/Y/Z datasets ('so that instances across datasets are not sychronized by dimension or subject'), not as a description of the train/test policy; no subject id is released, and the 300/700 split is exactly 3x100 / 7x100 of the 10 subjects x 10 gestures x 10 repetitions, so a subject-disjoint split is numerically consistent with the released sizes R. Subject sharing is therefore not established; a gesture interface for consumer devices is used by people it was not trained on I.
Label
as AllGestureWiimoteX. The gesture classes are the collectors' protocol factor, but the gesture-recognition task was posed by the archive editor; the collectors' target was user identity R. Subject sharing across the split is not established (no ids; the sizes are consistent with a subject-disjoint split) R.
Open questions
Subject sharing inferred from the source design (R) but not documented, so Unclear rather than Concern.
real task projectile-point class (Avonlea, Clovis, Mix)
Archive task
projectile-point class (Avonlea, Clovis, Mix)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New physical objects
Exact duplicates
none found D
Why
Each case is an outline of an arrowhead. The source assigned outlines to a 36/175 split at random. We cannot check whether multiple views or representations of the same object were kept together.
Source
Classification of stone projectile points (arrowheads), a standard task in archaeology/anthropology: points are assigned to traditions on the basis of shape, which in turn indexes the group that made them and the period they were in use.
Publication
Ye & Keogh, "Time Series Shapelets: A New Primitive for Data Mining", KDD 2009, Sec. 5.2 (read); the authors also reference their own extended review of the projectile-point literature.
Full record
Benchmark task
Classify a length-251 outline series as Avonlea, Clovis or Mix.
The benchmark label is the target the source study posed (projectile-point class (Avonlea, Clovis, Mix)); label origin: source task.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split = the random 36/175 train/test split of Ye & Keogh 2009, Sec. 5.2 S; whether points from one site or object fall on both sides is not documented S
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New physical objects.
Split evidence
random 36/175 split by Ye & Keogh S; whether outlines of one artefact or site fall on both sides is undocumented.
BeefSPECTRO · Spectroscopy & chemistry · 30/30 train/test · length 470 · 5 classesreal tasksource task labelUnclearassumed new case: new physical samplesordered, non-temporalwavelength or wavenumber
Taskreal task
Source task
real task sample type - pure beef vs beef adulterated with one of four offals (heart, tripe, kidney, liver)
Archive task
sample type - pure beef vs beef adulterated with one of four offals (heart, tripe, kidney, liver)
Label attached by
Source studysource taskSR
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New physical samples
Exact duplicates
none found D
Why
The data are spectral measurements of beef samples. It is not documented whether repeated measurements of one physical sample were kept in the same set. Without that information, performance on new samples cannot be distinguished from performance on further measurements of familiar samples.
Source
A food-authenticity study from the Analytical Sciences Unit of the Institute of Food Research, Norwich (Kemsley's group): detect adulteration of beef with cheaper offal.
Publication
Al-Jowder, Kemsley, Wilson, "Detection of adulteration in cooked meat products by mid-infrared spectroscopy", J. Agric. Food Chem. 50(6), 1325-1329, 2002, doi:10.1021/jf0108967. Only the abstract was read (via the Europe PMC REST API); the ACS full text is paywalled, so the mapping from class index to adulterant and the choice of cooking regime could not be checked.
Full record
Benchmark task
Classify a univariate length-470 mid-IR spectrum into one of five beef/offal sample types.
The benchmark label is the target the source study posed (sample type - pure beef vs beef adulterated with one of four offals (heart, tripe, kidney, liver)); label origin: source task.
Data origin
ordered, non-temporalindex: wavelength or wavenumber
Archive changes
spectrum-as-series
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New physical samples.
Split evidence
60 spectra of five sample types; whether replicate spectra of one sample straddle the split is undocumented R.
real task Shape retrieval over 70 object classes (MPEG-7 CE-Shape-1 descriptor benchmark); no applied problem
Archive task
beetle vs fly, a two-class pair carved out of the source's 70 classes
Label attached by
CuratorsredefinedS
Verdict
failstask replacement
The source posed shape retrieval scored by class membership over its 70 object classes; the collection was assembled to compare shape descriptors and has no applied stakeholder, as for ShapesAll, Car and Plane.
SplitUnclear
New case assumed
New source images
Exact duplicates
none found D
Why
The cases are outlines extracted from images of beetles and flies. We do not know how the original images were assigned to training and testing. If different versions of one image occur in both sets, the model could recognise a familiar shape. Such shared images have not been verified.
Source
The images come from MPEG-7 Core Experiment CE-Shape-1 Part B, a database built to benchmark 2D shape descriptors for the MPEG-7 standard - not to solve an applied recognition problem.
Publication
Latecki, Lakamper & Eckhardt, "Shape Descriptors for Non-rigid Shapes with a Single Closed Contour", CVPR 2000 (read), which reports the MPEG-7 Core Experiment CE-Shape-1 that produced the image database; the outlines were extracted and mapped to series by Hills & Bagnall at UEA (shapelet-transform work), whose paper was not reachable in full text.
Full record
Benchmark task
Classify a length-512 outline series as beetle or fly.
Who would use it
none applied; the collection exists to evaluate shape descriptors for the MPEG-7 standard, and this two-class subset exists to test TSC algorithms
Task construction
failtask replacement
Benchmark target: beetle vs fly, a two-class pair carved out of the source's 70 classes. Source target or purpose: Shape retrieval over 70 object classes (MPEG-7 CE-Shape-1 descriptor benchmark); no applied problem. Label attached by: Curators. Mechanism 'task replacement': the archive replaces the source's label with a different task, or defines classes by its own rule where the source posed none; here the curators selected two similar classes from the source pool as a hard pair (subsetting).
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
outlines drawn from the same MPEG-7 CE-Shape-1 image pool (instance sharing unverified)
Overall verdict
fails task construction (leakage unknown)
New case assumed
New source images.
Split evidence
20 distinct MPEG-7 shapes per class; nothing documents shared source images across the split R.
Label
the archive page states that the curators extracted the outlines and defined the two-class problem S; that the pair was chosen for its similarity is our reading R. The class names are the source's, the task is the curators'.
Open questions
Subsetting of a source class set: the two class names come from MPEG-7, the two-class task from the curators; the source protocol was retrieval, not classification.
real task Shape retrieval over 70 object classes (MPEG-7 CE-Shape-1 descriptor benchmark); no applied problem
Archive task
bird vs chicken, a two-class pair carved out of the source's 70 classes
Label attached by
CuratorsredefinedS
Verdict
failstask replacement
The source posed shape retrieval scored by class membership over its 70 object classes; the collection was assembled to compare shape descriptors and has no applied stakeholder, as for ShapesAll, Car and Plane.
SplitUnclear
New case assumed
New source images
Exact duplicates
none found D
Why
Same as BeetleFly. [BeetleFly: The cases are outlines extracted from images of beetles and flies. We do not know how the original images were assigned to training and testing. If different versions of one image occur in both sets, the model could recognise a familiar shape. Such shared images have not been verified.]
Source
The images come from MPEG-7 Core Experiment CE-Shape-1 Part B, a database built to benchmark 2D shape descriptors for the MPEG-7 standard - not to solve an applied recognition problem.
Publication
Latecki, Lakamper & Eckhardt, "Shape Descriptors for Non-rigid Shapes with a Single Closed Contour", CVPR 2000 (read), which reports the MPEG-7 Core Experiment CE-Shape-1 that produced the image database; the outlines were extracted and mapped to series by Hills & Bagnall at UEA (shapelet-transform work), whose paper was not reachable in full text.
Full record
Benchmark task
Classify a length-512 outline series as bird or chicken.
Who would use it
none applied; the collection exists to evaluate shape descriptors for the MPEG-7 standard, and this two-class subset exists to test TSC algorithms
Task construction
failtask replacement
Benchmark target: bird vs chicken, a two-class pair carved out of the source's 70 classes. Source target or purpose: Shape retrieval over 70 object classes (MPEG-7 CE-Shape-1 descriptor benchmark); no applied problem. Label attached by: Curators. Mechanism 'task replacement': the archive replaces the source's label with a different task, or defines classes by its own rule where the source posed none; here the curators selected two similar classes from the source pool as a hard pair (subsetting).
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
outlines drawn from the same MPEG-7 CE-Shape-1 image pool (instance sharing unverified)
Overall verdict
fails task construction (leakage unknown)
New case assumed
New source images.
Split evidence
20 distinct MPEG-7 shapes per class; nothing documents shared source images across the split R.
Label
the archive page states that the curators extracted the outlines and defined the two-class problem S; that the pair was chosen for its similarity is our reading R. The class names are the source's, the task is the curators'.
Open questions
Subsetting of a source class set: the two class names come from MPEG-7, the two-class task from the curators; the source protocol was retrieval, not classification.
toy task None; unpublished generator (Grenoble LIG/AMA)
Archive task
Bell at begin, none, or at end of a central plate
Label attached by
Generatorsource taskS
Verdict
failstoy source task
The source data were staged or generated to demonstrate a method.
SplitGenerated
New case assumed
New generated examples
Exact duplicates
none found D
Why
This is a generated task rather than measurements from people, devices or specimens. The corresponding source-separation question is therefore not assessed here. Its split would instead need to be examined through the generation procedure; synthetic origin alone is not evidence of leakage.
Source
No real-world problem exists. BME (Begin-Middle-End) is a synthetic three-class benchmark: every series is a central plate (positive or negative) and the class is decided by whether a small positive bell appears at the start of the series, nowhere, or at the end.
Publication
No publication found. The only provenance is the donor line on the archive page: "Data source: University Joseph Fourier (Grenoble I), Laboratoire d informatique de Grenoble(LIG), Equipe AMA" (timeseriesclassification.com, BME; donated_by "Joseph Fourier University"), with no data_source_link. The most likely origin is the Grenoble AMA group of A. Douzal-Chouakria (candidate: Frambourg, Douzal-Chouakria & Gaussier, "Learning Multiple Temporal Matching for Time Series Classification", IDA 2013), but that full text could not be reached (hal.science/hal-00908431 returns an access-denied page, ResearchGate returns HTTP 403) and no source consulted states that BME was introduced there. Nothing about the generator (bell amplitude, width, noise model) is documented.
Full record
Benchmark task
Classify a univariate length-128 series as Begin, Middle or End.
Who would use it
none (synthetic demo)
Task construction
failtoy source task
Benchmark target: Bell at begin, none, or at end of a central plate. Source target or purpose: None; unpublished generator (Grenoble LIG/AMA). Label attached by: Generator. Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question.
Data origin
syntheticindex: generator step
Archive changes
simulation
Leakage
n/a (generated)not assessed (generated data)
Split sources
Archive default split; who built it and by what rule is not documented RS
the same donor line and the same bell-at-the-edge generator design
Overall verdict
fails task construction (generated data)
New case assumed
New generated examples.
Split evidence
generated data: no deployment unit.
Label
generated data: the generator's class is the only label. No publication; the only provenance is the donor line on the archive page (Grenoble LIG/AMA); the class is the generator's R.
CarSENSOR · Shapes & object images · 60/60 train/test · length 577 · 4 classesreal tasksource task labelUnclearassumed new case: new vehicles or recordingsordered, non-temporalcontour arc-lengthfamily: thakoor-silhouettes
Taskreal task
Source task
real task vehicle body type (sedan, pickup, minivan, SUV)
Archive task
vehicle body type (sedan, pickup, minivan, SUV)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New vehicles or recordings
Exact duplicates
none found D
Why
The outlines were extracted from traffic videos. Several frames can show the same vehicle under very similar conditions. The documentation does not establish whether all frames of a vehicle or video were kept in one set, so performance on new vehicles or capture occasions cannot be checked.
Source
A computer-vision shape-classification study. Thakoor & Gao needed pairs of *similar* shape classes to test a discriminative HMM shape classifier, and built two purpose-made databases; the second was vehicle silhouettes segmented out of outdoor traffic video by a motion-based object extractor.
Publication
Thakoor & Gao, "Shape Classifier based on Generalized Probabilistic Descent Method with Hidden Markov Descriptor", ICCV 2005, Sec. 7.2 (read in full, CiteSeerX copy): "In the second experiment, we classified the vehicles shapes extracted from traffic videos using the motion information... The vehicles were classified into one of the four class: sedan, pickup, minivan or SUV." Same section: "Videos were captured at resolution of 320 x 240. As object extraction approach used does not deal with shadows, the extracted car shapes are distorted in the bottom half due to shadow. 30 samples for each class were extracted from the video." The companion database page (biomecis.uta.edu/shape_data.htm) states the shapes are stored as "Cartesian coordinates of each point on the perimeter of the shape".
Full record
Benchmark task
Classify a univariate length-577 series (an unrolled vehicle outline) into one of four body types.
Who would use it
traffic-surveillance / intelligent-transportation vision systems that classify vehicle type from a roadside or overhead camera
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (vehicle body type (sedan, pickup, minivan, SUV)); label origin: source task.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
CBFSIMULATED · Simulated · 30/900 train/test · length 128 · 3 classesfailstoy source taskGeneratedassumed new case: new generated examplessyntheticgenerator step
Taskfails
Source task
toy task None; demo signal for local feature extraction
Archive task
Cylinder, bell or funnel shape in noise
Label attached by
Generatorsource taskR
Verdict
failstoy source task
The source data were staged or generated to demonstrate a method.
SplitGenerated
New case assumed
New generated examples
Exact duplicates
none found D
Why
This is a generated task rather than measurements from people, devices or specimens. The corresponding source-separation question is therefore not assessed here. Its split would instead need to be examined through the generation procedure; synthetic origin alone is not evidence of leakage.
Source
No real-world problem exists. Cylinder-Bell-Funnel is a synthetic benchmark defined by Naoki Saito in his PhD thesis to demonstrate local feature extraction: each series is standard normal noise plus a class-specific offset shape (plateau, ramp-up, ramp-down) at a random location and width.
Publication
Saito, N., "Local Feature Extraction and Its Applications Using a Library of Bases", PhD thesis, Yale University, 1994
Full record
Benchmark task
Classify a univariate length-128 series as cylinder, bell, or funnel.
Who would use it
none (synthetic demo)
Task construction
failtoy source task
Benchmark target: Cylinder, bell or funnel shape in noise. Source target or purpose: None; demo signal for local feature extraction. Label attached by: Generator. Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question.
Data origin
syntheticindex: generator step
Archive changes
simulation
Leakage
n/a (generated)not assessed (generated data)
Split sources
Archive default split; who built it and by what rule is not documented RR
ChinatownTRAFFIC · Traffic & pedestrians · 20/343 train/test · length 24 · 2 classesfailsmetadata as labelRelated periodsassumed new case: new days or later periodstemporaltimefamily: melbourne-pedestrianopen question
Taskfails
Source task
none None; Melbourne pedestrian counts
Archive task
Weekend vs weekday
Label attached by
CuratorsmetadataS
Verdict
failsmetadata as label
The source collected the data but posed no classification target.
SplitRelated periods
New case assumed
New days or later periods
Exact duplicates
none found D
Why
A test day may be close to a training day and share its season or local events. These are different days of one process, so this is not automatically leakage. No source says whether the split holds out a later period that follows all the training days.
Source
The City of Melbourne runs an automated pedestrian counting system: sensors at fixed sites count people passing, and the counts are published hourly as open data.
Publication
City of Melbourne Pedestrian Counting System open data, read at https://data.melbourne.vic.gov.au/explore/dataset/pedestrian-counting-system-monthly-counts-per-hour/ -- the system "helps to understand how people use different city locations at different times of day to better inform decision-making and plan for the future", and the published dataset contains "hourly pedestrian counts since 2009 from pedestrian sensor devices located across the city". The UCR dataset was cut from the 2017 data by Hoang Anh Dau; no paper describes the curation, only the archive description page.
Full record
Benchmark task
Classify a 24-point hourly pedestrian-count profile from the Chinatown-Swanston St (North) sensor as a weekend day or a weekday, training on 20 labelled days.
Who would use it
City planners, transport and retail analysts use these footfall demand curves; the weekend/weekday label itself is never unknown in practice (the date is recorded), so the benchmark task is a proxy rather than an operational one.
Task construction
failmetadata as label
Benchmark target: Weekend vs weekday. Source target or purpose: None; Melbourne pedestrian counts. Label attached by: Curators. Mechanism 'metadata as label': the archive derives the classes from a recording attribute.
Data origin
temporalindex: time
Archive changes
windowing
Leakage
pass (related periods)different periods of one ongoing process
Split sources
Archive default split; who built it and by what rule is not documented RSR
Chinatown is one sensor and year of MelbournePedestrian
Overall verdict
fails task construction
New case assumed
New days or later periods.
Split evidence
every series is one day of the single Chinatown-Swanston St (North) sensor in 2017, as the archive page states S; how days were assigned to train and test is not documented, and adjacent days share season and local events R.
Label
weekend/weekday classes defined from each day's date on the archive page, which credits H.A. Dau as data editor; the City of Melbourne collected counts, not classes S.
Open questions
Related periods (different days, same sensor and year); whether this counts as clean is undecided.
none None; EPANET simulation built to test missing-value imputation
Archive task
Three undocumented classes
Label attached by
CuratorsredefinedR
Verdict
failstask replacement
The source (Li et al., KDD 2009) simulated the data with EPANET to evaluate a missing-value imputation and summarization method; it posed no classification target.
SplitConcern
New case assumed
New simulation runs
Exact duplicates
none found D
Why
The audit describes cases taken at nearby times in one simulated system. A training case and test case can therefore describe almost the same system conditions. The result tests more cases from that simulation. It does not show whether the model works on a new simulation run; similarity between cases would need a separate check.
Source
EPANET simulation of chlorine concentration at every node of a drinking-water network over 15 simulated days, produced by Li et al. (KDD 2009) to evaluate a missing-value imputation method; no domain classification target was ever defined for it.
Publication
Li, McCann, Pollard, Faloutsos, "DynaMMo - Mining and Summarization of Coevolving Sequences with Missing Values", KDD 2009, Sec. 4.1.2 Datasets (read) - "Chlorine Dataset (Chlorine) - The Chlorine dataset (see sample in Figure 6(a)) was produced by EPANET 2 that models the hydraulic and water quality behavior of water distribution piping systems. ... The data set consists of 166 nodes (pipe junctions) and measurement of the Chlorine concentration level at all these nodes during 15 days (one measurement for every 5 minutes, a total of 4310 time ticks). Since the water demand pattern during the 15 days follows a clear global periodic pattern (daily cycle, dominating residential demand pattern), EPANET would correctly reflect the pattern in the Chlorine concentration with a few exceptions and slight time shifts." The archive page attributes the data to Lei Li's CMU PhD thesis; both links it gives (cs.cmu.edu/~leili/pubs/leili-thesis.pdf and .../li-kdd09.pdf) are dead (HTTP 404), and the thesis itself was not reached - the KDD'09 paper above is the same data described by the same authors.
Full record
Benchmark task
Classify a univariate length-166 series (a cross-section of chlorine concentration across the 166 network nodes) into three undocumented classes.
Who would use it
water utilities / EPANET modellers care about residual chlorine across a network; but nobody outside the archive poses the shipped 3-class problem, so the benchmark task has no identified user.
Task construction
failtask replacement
Benchmark target: Three undocumented classes. Source target or purpose: None; EPANET simulation built to test missing-value imputation. Label attached by: Curators. Mechanism 'task replacement': the curators imposed three classes, of undocumented meaning, on simulation output that had no labels.
Data origin
ordered, non-temporalindex: entity index (network node)
The shipped series runs over the 166 network nodes at one time tick (the source matrix transposed); the archive text describes it as 15 days of 5-minute measurements.
Archive changes
simulationz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Archive default split; who built it and by what rule is not documented RSR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails task construction and leakage
New case assumed
New simulation runs.
Split evidence
train and test are cross-sections at adjacent time ticks of one 15-day simulation run; adjacent ticks are near-duplicates R.
Label
three classes imposed by curation over EPANET output; the source study had no labels and the classes are undocumented R. Series length 166 equals the EPANET node count, so the shipped index is network node, not time (the archive text describes 15 days of 5-minute measurements) R; the three classes were imposed by the curators with undocumented meaning R.
Open questions
The source posed no classification target; what the three archive classes mean, how they were assigned, and how the simulation ticks were allocated to train and test are undocumented in every source consulted.
The source study posed its own prediction task, which the archive did not keep as the label.
SplitConcern
New case assumed
New recording occasions
Exact duplicates
none found D
Why
The task is to identify four people from their ECG measurements. The archive defines each class as one person, so training and test share those people, and very likely one recording each. The same identities are expected in an identification task, but new recordings are not tested. The model may recognise characteristics of the recording as well as the person.
Source
The recordings come from a PhysioNet/Computers in Cardiology challenge on body-surface potential mapping (BSPM).
Publication
PhysioNet/Computing in Cardiology Challenge 2007, "Electrocardiographic Imaging of Myocardial Infarction" (https://moody-challenge.physionet.org/2007/), read 2026-08. Challenge page: "The BSPM data, consisting of ECG data for 352 torso-surface sites, are provided in easy-to-read ASCII format for a single averaged PQRST complex sampled at 2kHz."
Full record
Benchmark task
Given one length-1639 torso-surface ECG series, say which of four individuals it was recorded from.
Benchmark target: Which of four people. Source target or purpose: Infarct extent and location. Label attached by: Curators. Mechanism 'instance identity as label': the archive uses the recording's source (subject, sensor or actor) as the class.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
40 train / 1380 test over four people; built by the curators from the challenge cases; rule undocumented RSR
Multichannel source
yes, one channel from a multichannel recording
The source recorded body-surface potential map, one averaged PQRST complex; the UCR release keeps a single channel of it. The spatial structure that made the original problem solvable is destroyed: electrocardiographic imaging inverts the *whole* 352-site map onto the heart, whereas the archive presents each site as an independent univariate series with no electrode-position metadata. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
Dataset family
none from the same source task
Overall verdict
fails task construction and leakage
New case assumed
New recording occasions.
Split evidence
the archive defines the four classes as four people, so each person is on both sides S; that a person's series share one session and are electrodes on the same torso at the same moment R.
Label
person identity, stated by the archive S, derived by the curators from the challenge's case grouping; the challenge target was infarct extent R. The identification of the source as the PhysioNet/CinC 2007 challenge (infarct extent and location, four cases) is our inference from the archive's 'one of the Computers in Cardiology challenges' R; under it each series is one torso-site average, not a separate recording R.
Open questions
The 2007 challenge identification is our inference; the archive page says only 'one of the Computers in Cardiology challenges'.
CoffeeSPECTRO · Spectroscopy & chemistry · 28/28 train/test · length 286 · 2 classesreal tasksource task labelUnclearassumed new case: new physical samplesordered, non-temporalwavelength or wavenumber
Taskreal task
Source task
real task coffee variety (Arabica vs Robusta)
Archive task
coffee variety (Arabica vs Robusta)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New physical samples
Exact duplicates
none found D
Why
The cases are spectral measurements of coffee. We do not know whether a sample was measured more than once and whether those measurements were kept together. If one measurement is in training and another in testing, the test sample is familiar. The missing sample information prevents us from checking this possibility.
Source
A chemometrics study in food quality assurance: distinguish the two main commercial coffee species, Arabica and Robusta, in instant coffee using Fourier-transform infrared (FTIR) spectroscopy, as a cheap alternative to wet-chemistry authentication (Arabica commands higher prices, so substitution fraud is the practical concern).
Publication
Briandet, Kemsley, Wilson, "Discrimination of Arabica and Robusta in Instant Coffee by Fourier Transform Infrared Spectroscopy and Chemometrics", J. Agricultural and Food Chemistry, 1996
Full record
Benchmark task
Classify a univariate length-286 series as Arabica or Robusta.
Who would use it
food-authenticity testing labs / regulators
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (coffee variety (Arabica vs Robusta)); label origin: source task.
Data origin
ordered, non-temporalindex: wavelength or wavenumber
Archive changes
spectrum-as-series
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New physical samples.
Split evidence
56 spectra; whether replicate acquisitions of one sample straddle the split is undocumented R.
ComputersDEVICE · Energy & appliances · 250/250 train/test · length 720 · 2 classesreal tasksource task labelUnclearassumed new case: new appliances or householdstemporaltimefamily: household-electricity-surveyopen question
Taskreal task
Source task
real task Household electricity monitoring for energy policy (Household Electricity Survey); each metered channel is annotated with its appliance, but the survey posed no classification task
Archive task
which computing device produced the day's consumption trace (desktop vs laptop)
Label attached by
Collectorssource taskSR
Verdict
real task
The survey's installing electricians annotated each monitored product with its appliance ("selected from a dropdown list"), so under policy L3 we count appliance-type recognition as the source's own task although the survey trained no classifier; the classification problem was posed on those annotations by the curators (Lines & Bagnall, SDM 2014; Lines 2015 thesis). Each input is an appliance trace metered on its own, so the task is appliance recognition, not whole-house disaggregation.
SplitUnclear
New case assumed
New appliances or households
Exact duplicates
none found D
Why
The source measured each physical appliance over many days. We do not know whether all days from an appliance or household were kept in one set. If the same appliance appears in both sets, the model may recognise its usual pattern. That would be a narrower test than recognising appliance types in new homes.
Source
These series come from the UK government-sponsored household electricity monitoring study known as "Powering the Nation" (the DECC/Defra/Energy Saving Trust Household Electricity Survey).
Publication
UK Household Electricity Survey (DECC, Defra and the Energy Saving Trust, 2010-2011); Intertek Report R66141, "Household Electricity Survey: A study of domestic electrical product usage", final report issue 4. Household counts differ: the archive page, the thesis and Intertek (p. 24) give 251 households, Report 5 (p. 4) and gov.uk give 250; the 26 year-long households were sampled every 10 minutes except from late November 2010 to February 2011, when they recorded every 2 minutes (Intertek p. 64). Curated into TSC datasets by Lines & Bagnall (UEA): J. Lines, PhD thesis, UEA, 2015, Sec. 3.2.2 and Table 3.3 ("We provide five new classification problems that focus on domestic appliance classification from electricity consumption"); introduced in Lines & Bagnall, "Ensembles of elastic distance measures for time series classification", SIAM SDM 2014 (not read: no full text found).
Full record
Benchmark task
Classify a 720-point daily consumption profile as a desktop or a laptop computer.
Who would use it
UK energy policy (the survey); the curators' stated aim was to extract more from smart-meter data by "classifying appliances according to their electricity usage profiles". The derived task (naming a device from its own metered daily profile) is appliance recognition, not disaggregation of a whole-house load.
Task construction
passnone (source task label)
The benchmark label is the survey's appliance annotation of the metered channel (which computing device produced the day's consumption trace (desktop vs laptop)); label origin: source task, by the convention that a collectors' annotation of the quantity a deployed model would predict counts as the source task.
Data origin
temporalindex: time
Archive changes
windowingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split, made by the curators (Lines & Bagnall), who say these datasets were "split in a certain manner to avoid introducing bias" from one device appearing in both train and test S; the rule used is not stated and the files carry no household or device identifiers RIR
five appliance-group tasks cut from the UK Household Electricity Survey (ElectricDevices comes from a different study, the Visual Energy Trial); whether they share households or appliances is not documented
Overall verdict
real task; leakage unknown
New case assumed
New appliances or households.
Split evidence
each series is one day cut from a month (or, for some households, a year) of metering, so one appliance can supply many series R; which households, appliances and days entered the dataset is not documented. The curators say the split was made to avoid device bias, but give no rule, and the files carry no device identifiers, so device- or household-disjointness is neither documented nor refuted and sharing is not counted as leakage R. An appliance-recognition model meeting unseen appliances is the deployment case I.
Open questions
Appliance-type recognition is counted as the source task by convention (the survey annotated the channels but posed no classification task); a stricter reading files it as metadata. Appliance/household sharing inferred from the survey design but not documented, so Unclear rather than Concern.
CricketXHAR · Human motion & gesture · 390/390 train/test · length 300 · 12 classesreal tasksource task labelUnclearassumed new case: new peopletemporaltimesingle channelfamily: cricket
Taskreal task
Source task
real task which of twelve cricket umpire signals was performed
Archive task
which of twelve cricket umpire signals was performed
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New people
Exact duplicates
none found D
Why
The data contain repeated gestures, but the files do not say which person produced each case. We cannot check whether all gestures from a person were kept in one set. A test gesture might come from someone already represented in training. Performance on people the model has never seen is therefore not established.
Source
Automatic annotation of cricket matches from the officials' body motion: umpires signal match events (four, six, wide, no-ball, out, ...) with codified arm gestures, and the study instrumented umpires with wrist-worn accelerometers so that these signals could be recognised online and used to label the video/scorecard.
Publication
Ko, West, Venkatesh, Kumar, "Online Context Recognition in Multisensor Systems using Dynamic Time Warping", ISSNIP 2005 (DOI 10.1109/ISSNIP.2005.1595593). Not read: the paper is closed access (Unpaywall reports oa_status "closed"; the archive's own second_link is a dead IEEE Xplore login URL, and the Semantic Scholar "open access" copy at dro.deakin.edu.au returns an empty file). Layer 1 above therefore rests on the archive description page, the UCR briefing document and the UEA archive paper's description of the multivariate version.
Full record
Benchmark task
Classify a normalized length-300 single-axis wrist-accelerometer series as one of twelve cricket umpire signals.
Who would use it
sports-video annotation and automatic scoring systems; more generally the source authors' interest was online context recognition from wearable multisensor streams
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of twelve cricket umpire signals was performed); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selectionz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded acceleration from wrist-worn accelerometers; the UCR release keeps a single channel of it. Five of the six original channels are dropped (both wrists x three axes reduced to one axis), so gestures that differ mainly in the other arm's motion or in the plane of movement become harder or impossible to separate. Amplitude is removed by the donors' normalization, and the length normalization to 300 removes gesture duration. The problem the source posed is multivariate, and its place is the multivariate archive, where it already exists as Cricket.
CricketYHAR · Human motion & gesture · 390/390 train/test · length 300 · 12 classesreal tasksource task labelUnclearassumed new case: new peopletemporaltimesingle channelfamily: cricket
Taskreal task
Source task
real task which of twelve cricket umpire signals was performed
Archive task
which of twelve cricket umpire signals was performed
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New people
Exact duplicates
none found D
Why
Same as CricketX. [CricketX: The data contain repeated gestures, but the files do not say which person produced each case. We cannot check whether all gestures from a person were kept in one set. A test gesture might come from someone already represented in training. Performance on people the model has never seen is therefore not established.]
Source
Automatic annotation of cricket matches from the officials' body motion: umpires signal match events (four, six, wide, no-ball, out, ...) with codified arm gestures, and the study instrumented umpires with wrist-worn accelerometers so that these signals could be recognised online and used to label the video/scorecard.
Publication
Ko, West, Venkatesh, Kumar, "Online Context Recognition in Multisensor Systems using Dynamic Time Warping", ISSNIP 2005 (DOI 10.1109/ISSNIP.2005.1595593). Not read: the paper is closed access (Unpaywall reports oa_status "closed"; the archive's own second_link is a dead IEEE Xplore login URL, and the Semantic Scholar "open access" copy at dro.deakin.edu.au returns an empty file). Layer 1 above therefore rests on the archive description page, the UCR briefing document and the UEA archive paper's description of the multivariate version.
Full record
Benchmark task
Classify a normalized length-300 single-axis wrist-accelerometer series as one of twelve cricket umpire signals.
Who would use it
sports-video annotation and automatic scoring systems; more generally the source authors' interest was online context recognition from wearable multisensor streams
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of twelve cricket umpire signals was performed); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selectionz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded acceleration from wrist-worn accelerometers; the UCR release keeps a single channel of it. Five of the six original channels are dropped (both wrists x three axes reduced to one axis), so gestures that differ mainly in the other arm's motion or in the plane of movement become harder or impossible to separate. Amplitude is removed by the donors' normalization, and the length normalization to 300 removes gesture duration. The problem the source posed is multivariate, and its place is the multivariate archive, where it already exists as Cricket.
CricketZHAR · Human motion & gesture · 390/390 train/test · length 300 · 12 classesreal tasksource task labelUnclearassumed new case: new peopletemporaltimesingle channelfamily: cricket
Taskreal task
Source task
real task which of twelve cricket umpire signals was performed
Archive task
which of twelve cricket umpire signals was performed
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New people
Exact duplicates
none found D
Why
Same as CricketX. [CricketX: The data contain repeated gestures, but the files do not say which person produced each case. We cannot check whether all gestures from a person were kept in one set. A test gesture might come from someone already represented in training. Performance on people the model has never seen is therefore not established.]
Source
Automatic annotation of cricket matches from the officials' body motion: umpires signal match events (four, six, wide, no-ball, out, ...) with codified arm gestures, and the study instrumented umpires with wrist-worn accelerometers so that these signals could be recognised online and used to label the video/scorecard.
Publication
Ko, West, Venkatesh, Kumar, "Online Context Recognition in Multisensor Systems using Dynamic Time Warping", ISSNIP 2005 (DOI 10.1109/ISSNIP.2005.1595593). Not read: the paper is closed access (Unpaywall reports oa_status "closed"; the archive's own second_link is a dead IEEE Xplore login URL, and the Semantic Scholar "open access" copy at dro.deakin.edu.au returns an empty file). Layer 1 above therefore rests on the archive description page, the UCR briefing document and the UEA archive paper's description of the multivariate version.
Full record
Benchmark task
Classify a normalized length-300 single-axis wrist-accelerometer series as one of twelve cricket umpire signals.
Who would use it
sports-video annotation and automatic scoring systems; more generally the source authors' interest was online context recognition from wearable multisensor streams
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of twelve cricket umpire signals was performed); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selectionz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded acceleration from wrist-worn accelerometers; the UCR release keeps a single channel of it. Five of the six original channels are dropped (both wrists x three axes reduced to one axis), so gestures that differ mainly in the other arm's motion or in the plane of movement become harder or impossible to separate. Amplitude is removed by the donors' normalization, and the length normalization to 300 removes gesture duration. The problem the source posed is multivariate, and its place is the multivariate archive, where it already exists as Cricket.
real task land-cover / crop class of a ground pixel (24 classes)
Archive task
land-cover / crop class of a ground pixel (24 classes)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitConcern
New case assumed
New fields, scenes or seasons
Exact duplicates
2,753 of 16,800 test series (16.4%) exactly match a training series D
Why
The cases are pixel sequences from one scene and season. Nearby pixels may describe the same field and growing conditions. The value check also finds 2,753 of 16,800 test cases identical to training cases. This makes the test different from evaluating new fields, scenes or seasons.
Source
Land-cover / crop-type mapping from satellite image time series. Tan, Webb & Petitjean (SDM 2017, Sec. on SITS classification) describe the source data: "In this work, we used 46 geometrically and radiometrically corrected images taken by FORMOSAT-2 satellite...
Publication
Tan, Webb & Petitjean, "Indexing and classifying gigabytes of time series under time warping", SDM 2017 (read; the archive's data_source_link bit.ly/SDM2017 resolves to the figshare project for this paper). The related TempCNN paper (Pelletier, Webb & Petitjean, arXiv:1811.10166, Sec. 3.1-3.2) describes the same imagery: "The satellite dataset is composed of 46 Formosat-2 images acquired at 8 meter spatial resolution during the year 2006", with reference data from "farmer's declaration from 2006... ground field campaigns performed during 2006... and a reference map obtained with a semi-automatic procedure" (that paper aggregates to 13 classes rather than 24).
Full record
Benchmark task
Classify a length-46 per-pixel reflectance profile into one of 24 land-cover classes.
Who would use it
agricultural and environmental agencies producing land-cover maps (the data were released with permission of CesBIO and the French Space Agency)
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (land-cover / crop class of a ground pixel (24 classes)); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selectionz-normalization
Leakage
failexact cross-split matches (2753 of 16800 test series)
Split sources
Tan et al. (2017) release, 7200 train / 16800 test pixels of one scene and season; assignment rule undocumented RDR
Multichannel source
yes, one channel from a multichannel recording
The source recorded sequence of satellite reflectance observations for one 8 m ground pixel over a growing season; the UCR release keeps a single channel of it. Spatial context around the pixel is dropped (each pixel is treated independently), the multi-band signal is reduced to one channel, and z-normalization removes the absolute reflectance level, which is physically informative for land cover. The temporal axis itself is preserved intact. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
Dataset family
none from the same source task
Overall verdict
fails leakage
New case assumed
New fields, scenes or seasons.
Split evidence
all pixels come from one 64 km2 scene and one season; neighbouring pixels of one field are near-duplicates and 2753 of 16800 test pixels are identical to training pixels; a land-cover model is deployed on new fields and years R+D.
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New biological sources
Exact duplicates
none found D
Why
The sources explain size variation within each class by successive generations of a clone, but do not say how the 16/306 split was drawn or whether train and test come from the same clone. If they do, a new specimen from a familiar line is a narrower test than one from an independently collected source; performance on new biological sources is therefore not established.
Source
Part of the DIADIST project ("Diatom and Desmid Identification by Shape and Texture", BBSRC Bioinformatics Programme grant 754/BIO14262, Cardiff University with the Royal Botanic Garden Edinburgh), whose stated aim is "to investigate methods to enable the visual indexing of images and drawings of biological specimens held in a…
Publication
DIADIST project pages, Royal Botanic Garden Edinburgh (read: project description and "Diatom size reduction image sets for shape and appearance models"). Who converted the outlines to series, and how, is not documented - the archive page lists no donor.
Full record
Benchmark task
Classify a length-345 outline series into one of four diatom taxa.
Who would use it
diatomists and ecological/water-quality monitoring, where size reduction within a clone makes automated identification unreliable
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (diatom taxon (Gomphonema augur, Fragilariforma bicapitata, Stauroneis smithii, Eunotia tenella)); label origin: source task.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New biological sources.
Split evidence
within-class clone dependence is suggested by the card, but clone sharing across train and test is not documented; deployment on independently collected specimens is untested R.
DistalPhalanxOutlineAgeGroupIMAGE · Medical imaging · 400/139 train/test · length 80 · 3 classesreal tasksource task labelUnclearassumed new case: new patientsordered, non-temporalcontour arc-lengthfamily: phalanx-radiographsopen question
Taskreal task
Source task
real task chronological age of the subject (the study's continuous regression target)
Archive task
age group of the subject (0-6, 7-12, 13-19 years), three curator-defined bands
Label attached by
Curatorssource taskS
Verdict
real task
The source study modelled the subject's chronological age from bone outlines by regression, as a step towards automated bone age assessment, which paediatric radiology needs.
SplitUnclear
New case assumed
New patients
Exact duplicates
none found D
Why
The cases are finger-bone outlines extracted from X-rays. We do not know whether one patient contributed several images or whether those images were kept together. If one patient appears in both sets, the model may benefit from familiar anatomy. Patient identifiers are needed to check performance on new patients.
Source
All eleven "bone ageing" problems in the archive come from one project: Luke Davis's PhD "Predictive Modelling of Bone Ageing" at UEA, using hand radiographs from the Digital Hand Atlas (Cao et al.).
Publication
Bagnall & Davis, "Predictive Modelling of Bone Age through Classification and Regression of Bone Shapes", Technical Report CMPC14-02, arXiv:1406.4781 (read); images from Cao et al., "Digital hand atlas and web-based bone age assessment: system design and implementation" (not read; cited by the archive page and by Bagnall & Davis).
Full record
Benchmark task
Classify a length-80 distal-phalanx outline series into one of three age bands.
Who would use it
paediatric radiology - bone age assessment is performed daily in hospitals to check whether a child's skeletal development is on track
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (chronological age of the subject, the study's continuous regression target); label origin: source task. The curators coarsened the target into three bands (0-6, 7-12, 13-19 years); a coarsened source target still counts as source task.
the same ~1300 hand radiographs of the same subjects; different bones and labels (age band, Tanner-Whitehouse stage, segmentation correctness); PhalangesOutlinesCorrect pools three siblings
Overall verdict
real task; leakage unknown
New case assumed
New patients.
Split evidence
one bone outline per radiograph; nothing states whether a patient's radiographs can appear on both sides R.
Label
the archive page states the three age bands S; they discretise the source's continuous chronological-age regression target, set by the curators; treated as a coarsened source target, not a replacement R.
Open questions
Age discretised into three bands by the curators; we treat this as a coarsened source target rather than a replacement.
DistalPhalanxOutlineCorrectIMAGE · Medical imaging · 600/276 train/test · length 80 · 2 classesreal tasksource task labelUnclearassumed new case: new patientsordered, non-temporalcontour arc-lengthfamily: phalanx-radiographsopen question
Taskreal task
Source task
real task whether the automatic distal-phalanx outline extraction was judged correct (verification stage of the bone-ageing pipeline)
Archive task
Whether segmentation is judged correct
Label attached by
Source studysource taskS
Verdict
real task
The source pipeline posed outline verification as one of its stages; the archive keeps that target as the label.
SplitUnclear
New case assumed
New patients
Exact duplicates
none found D
Why
Same as DistalPhalanxOutlineAgeGroup. [DistalPhalanxOutlineAgeGroup: The cases are finger-bone outlines extracted from X-rays. We do not know whether one patient contributed several images or whether those images were kept together. If one patient appears in both sets, the model may benefit from familiar anatomy. Patient identifiers are needed to check performance on new patients.]
Source
All eleven "bone ageing" problems in the archive come from one project: Luke Davis's PhD "Predictive Modelling of Bone Ageing" at UEA, using hand radiographs from the Digital Hand Atlas (Cao et al.).
Publication
Bagnall & Davis, "Predictive Modelling of Bone Age through Classification and Regression of Bone Shapes", Technical Report CMPC14-02, arXiv:1406.4781 (read); images from Cao et al., "Digital hand atlas and web-based bone age assessment: system design and implementation" (not read; cited by the archive page and by Bagnall & Davis).
Full record
Benchmark task
Classify a length-80 distal-phalanx outline series as a correct or incorrect segmentation.
Who would use it
paediatric radiology - bone age assessment is performed daily in hospitals to check whether a child's skeletal development is on track
Task construction
passnone (source task label)
Benchmark target: Whether segmentation is judged correct. Source target or purpose: whether the automatic outline was judged correct (verification stage of the bone-ageing pipeline). Label attached by: Source study. The archive keeps the pipeline's own verification label, so task construction passes.
the same ~1300 hand radiographs of the same subjects; different bones and labels (age band, Tanner-Whitehouse stage, segmentation correctness); PhalangesOutlinesCorrect pools three siblings
Overall verdict
real task; leakage unknown
New case assumed
New patients.
Split evidence
one bone outline per radiograph; nothing states whether a patient's radiographs can appear on both sides R.
Label
the correct/incorrect label was produced by Davis & Bagnall (UEA) inside their own bone-ageing pipeline (arXiv:1406.4781) as its verification stage; it describes the segmentation, not the patient S.
Open questions
The evaluators' adjudication rule and agreement, and whether patients were kept apart across train and test, are undocumented.
DistalPhalanxTWIMAGE · Medical imaging · 400/139 train/test · length 80 · 6 classesreal tasksource task labelUnclearassumed new case: new patientsordered, non-temporalcontour arc-lengthfamily: phalanx-radiographs
Taskreal task
Source task
real task Tanner-Whitehouse skeletal maturity stage of the distal phalanx of the middle finger
Archive task
Tanner-Whitehouse skeletal maturity stage of the distal phalanx of the middle finger
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New patients
Exact duplicates
none found D
Why
Same as DistalPhalanxOutlineAgeGroup. [DistalPhalanxOutlineAgeGroup: The cases are finger-bone outlines extracted from X-rays. We do not know whether one patient contributed several images or whether those images were kept together. If one patient appears in both sets, the model may benefit from familiar anatomy. Patient identifiers are needed to check performance on new patients.]
Source
All eleven "bone ageing" problems in the archive come from one project: Luke Davis's PhD "Predictive Modelling of Bone Ageing" at UEA, using hand radiographs from the Digital Hand Atlas (Cao et al.).
Publication
Bagnall & Davis, "Predictive Modelling of Bone Age through Classification and Regression of Bone Shapes", Technical Report CMPC14-02, arXiv:1406.4781 (read); images from Cao et al., "Digital hand atlas and web-based bone age assessment: system design and implementation" (not read; cited by the archive page and by Bagnall & Davis).
Full record
Benchmark task
Classify a length-80 distal-phalanx outline series into one of 6 Tanner-Whitehouse stages.
Who would use it
paediatric radiology - bone age assessment is performed daily in hospitals to check whether a child's skeletal development is on track
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (Tanner-Whitehouse skeletal maturity stage of the distal phalanx of the middle finger); label origin: source task.
the same ~1300 hand radiographs of the same subjects; different bones and labels (age band, Tanner-Whitehouse stage, segmentation correctness); PhalangesOutlinesCorrect pools three siblings
Overall verdict
real task; leakage unknown
New case assumed
New patients.
Split evidence
one bone outline per radiograph; nothing states whether a patient's radiographs can appear on both sides R.
DodgerLoopDaySENSOR · Traffic & pedestrians · 78/80 train/test · length 288 · 7 classesfailsmetadata as labelRelated periodsassumed new case: new days or later periodstemporaltimefamily: dodger-loopopen question
Taskfails
Source task
real task Unsupervised detection of baseball-game traffic events (crowds leaving Dodger Stadium)
Archive task
Day of week
Label attached by
CuratorsmetadataSR
Verdict
failsmetadata as label
The source posed unsupervised detection of traffic events caused by baseball games (UCI: predicting the presence of a game), not day-of-week prediction.
SplitRelated periods
New case assumed
New days or later periods
Exact duplicates
none found D
Why
Days from one traffic detector are on both sides of the split; how they were assigned is not documented. Training and test days can be part of the same baseball home stand or share similar traffic conditions. The test contains different days, but does not specifically evaluate a later period or another detector. This relationship should be considered when interpreting the score.
Source
Ihler, Hutchins & Smyth collected freeway loop-detector counts to study *unsupervised event detection*: learning a time-varying Poisson model of normal traffic so that anomalous bursts (crowds leaving Dodger Stadium after a baseball game) can be flagged without being told when games occurred.
Publication
Ihler, Hutchins & Smyth, "Learning to Detect Events with Markov-Modulated Poisson Processes", ACM TKDD 1(3), Art. 13, 2007 (journal version of the KDD 2006 paper the archive cites; read in full), Sec. 3, p. 6: "The second dataset will be referred to as the freeway traffic data and consists of estimated vehicle counts every 5 minutes over 6 months from an inductive loop-sensor located on the Glendale on-ramp to the 101-North freeway in Los Angeles." Same page: "the baseball-related events at this on-ramp correspond to traffic leaving at the end of a game". The UCI "Dodgers Loop Sensor" page (read) confirms "Loop sensor data was collected for the Glendale on ramp for the 101 North freeway in Los Angeles", 288 five-minute observations per day over 25 weeks, and states the goal as predicting "the presence of a baseball game at Dodgers stadium".
Full record
Benchmark task
Given a 288-point 24-hour on-ramp traffic profile (with missing values), predict which day of the week it is.
Who would use it
traffic operations / event detection (Caltrans PeMS-style monitoring); for the day-of-week label specifically there is no real stakeholder, since the calendar already supplies the answer
Task construction
failmetadata as label
Benchmark target: Day of week. Source target or purpose: Unsupervised event detection; no class target. Label attached by: Curators. Mechanism 'metadata as label': the archive derives the classes from a recording attribute.
Data origin
temporalindex: time
Archive changes
windowing
Leakage
pass (related periods)different periods of one ongoing process
Split sources
Days of one loop detector assigned to train/test at random by the archive contributor (C. Yeh) RR
the same 158 daily windows of one loop-detector recording, labelled three ways
Overall verdict
fails task construction
New case assumed
New days or later periods.
Split evidence
158 consecutive days of one loop detector split at random; deployment on that ramp sees new days, but neighbouring days of the same home stand sit on both sides R.
Label
day-of-week label derived by the archive contributor (C. Yeh) from timestamps; Ihler et al. collected counts for event detection R. The day-of-week label was derived by the archive contributor from timestamps; the source study collected counts for unsupervised event detection R.
Open questions
Related periods: different days of one detector; not counted as leakage. Whether related periods should count as clean is undecided.
DodgerLoopGameSENSOR · Traffic & pedestrians · 20/138 train/test · length 288 · 2 classesfailsmetadata as labelRelated periodsassumed new case: new days or later periodstemporaltimefamily: dodger-loopopen question
Taskfails
Source task
real task Unsupervised detection of baseball-game traffic events (crowds leaving Dodger Stadium); the game schedule was validation ground truth
Archive task
whether a Dodgers home game took place on the recorded day
Label attached by
CuratorsmetadataSR
Verdict
failsmetadata as label
The source study posed unsupervised detection of stadium-crowd traffic events for traffic operations and validated it against the game schedule, which the archive uses as the label.
SplitRelated periods
New case assumed
New days or later periods
Exact duplicates
none found D
Why
Same as DodgerLoopDay. [DodgerLoopDay: Days from one traffic detector are on both sides of the split; how they were assigned is not documented. Training and test days can be part of the same baseball home stand or share similar traffic conditions. The test contains different days, but does not specifically evaluate a later period or another detector. This relationship should be considered when interpreting the score.]
Source
Same recording and same study as DodgerLoopDay: Ihler, Hutchins & Smyth modelled 5-minute vehicle counts from an inductive loop detector on the Glendale on-ramp of the 101-North freeway in Los Angeles, next to Dodger Stadium, in order to detect anomalous crowd events *without* being told when games happened.
Publication
Ihler, Hutchins & Smyth, "Learning to Detect Events with Markov-Modulated Poisson Processes", ACM TKDD 1(3), Art. 13, 2007 (journal version of the KDD 2006 paper the archive cites; read in full), Sec. 3, p. 6: "The second dataset will be referred to as the freeway traffic data and consists of estimated vehicle counts every 5 minutes over 6 months from an inductive loop-sensor located on the Glendale on-ramp to the 101-North freeway in Los Angeles." Same page, on the labels: "The lower panel of Figure 4 shows a set of known (ground-truth) events for this data (which are unknown to the model and only used for validation) corresponding to the dates and times of baseball games." And: "the baseball-related events at this on-ramp correspond to traffic leaving at the end of a game when large numbers of individuals leave the stadium and get on the freeway; thus, the event has a signature in the data that will tend to lag in time behind that of the baseball game itself."
Full record
Benchmark task
Given a 288-point 24-hour on-ramp traffic profile (with missing values), decide whether that day was a Dodgers home-game day.
Who would use it
traffic operations and event-aware routing; the underlying capability (detecting stadium egress from ramp counts) is genuinely useful, though the game schedule itself is public
Task construction
failmetadata as label
The benchmark label (home-game day) comes from the published game schedule attached to each date; the source used that schedule only as validation ground truth for unsupervised event detection; label origin: metadata.
Data origin
temporalindex: time
Archive changes
windowing
Leakage
pass (related periods)different periods of one ongoing process
Split sources
Days of one loop detector assigned to train/test at random by the archive contributor (C. Yeh) RR
the same 158 daily windows of one loop-detector recording, labelled three ways
Overall verdict
fails task construction
New case assumed
New days or later periods.
Split evidence
158 consecutive days of one loop detector split at random; deployment on that ramp sees new days, but neighbouring days of the same home stand sit on both sides R.
Label
label from the game schedule; the UCI source problem was detecting games from loop counts, so the label is the source's own target R. The game-day label comes from the Dodgers home-game schedule attached to the date; treated as the source's own target because the source study validated its unsupervised detector against that schedule R.
Open questions
Label from the game schedule attached to the date (calendar metadata) while the sibling DodgerLoopDay/Weekend labels are listed under metadata; we count the game label as the source's own target because Ihler et al. validated their event detector against the game schedule. Split: related periods (as DodgerLoopDay); whether this counts as clean is undecided.
DodgerLoopWeekendSENSOR · Traffic & pedestrians · 20/138 train/test · length 288 · 2 classesfailsmetadata as labelRelated periodsassumed new case: new days or later periodstemporaltimefamily: dodger-loopopen question
Taskfails
Source task
real task Unsupervised detection of baseball-game traffic events (crowds leaving Dodger Stadium); weekday/weekend rhythm modelled as background, not predicted
Archive task
Weekend vs weekday
Label attached by
CuratorsmetadataSR
Verdict
failsmetadata as label
The source posed unsupervised detection of traffic events caused by baseball games (validated against the game schedule); it treated the weekday/weekend rhythm as known background, not as a target.
SplitRelated periods
New case assumed
New days or later periods
Exact duplicates
none found D
Why
Same as DodgerLoopDay. [DodgerLoopDay: Days from one traffic detector are on both sides of the split; how they were assigned is not documented. Training and test days can be part of the same baseball home stand or share similar traffic conditions. The test contains different days, but does not specifically evaluate a later period or another detector. This relationship should be considered when interpreting the score.]
Source
Same recording and same study as DodgerLoopDay and DodgerLoopGame: Ihler, Hutchins & Smyth modelled 5-minute vehicle counts from an inductive loop detector on the Glendale on-ramp of the 101-North freeway in Los Angeles.
Publication
Ihler, Hutchins & Smyth, "Learning to Detect Events with Markov-Modulated Poisson Processes", ACM TKDD 1(3), Art. 13, 2007 (journal version of the KDD 2006 paper the archive cites; read in full), Sec. 3, p. 6: "The second dataset will be referred to as the freeway traffic data and consists of estimated vehicle counts every 5 minutes over 6 months from an inductive loop-sensor located on the Glendale on-ramp to the 101-North freeway in Los Angeles. Figure 4 shows the temporal pattern for a particular week starting with Sunday morning and ending Saturday night. The daily rhythms of traffic flow are clearly visible, as is the distinction between weekdays and weekends." The UCI "Dodgers Loop Sensor" page (read) confirms 288 five-minute observations per day over 25 weeks.
Full record
Benchmark task
Given a 288-point 24-hour on-ramp traffic profile (with missing values), decide whether it is a weekend day or a weekday.
Who would use it
traffic operations / demand profiling; for the weekend label itself there is no real stakeholder, since the calendar supplies the answer directly
Task construction
failmetadata as label
Benchmark target: Weekend vs weekday. Source target or purpose: unsupervised detection of baseball-game traffic events, with the weekday/weekend rhythm as background. Label attached by: Curators. Mechanism 'metadata as label': the archive derives the classes from a recording attribute.
Data origin
temporalindex: time
Archive changes
windowing
Leakage
pass (related periods)different periods of one ongoing process
Split sources
Days of one loop detector assigned to train/test at random by the archive contributor (C. Yeh) RR
the same 158 daily windows of one loop-detector recording, labelled three ways
Overall verdict
fails task construction
New case assumed
New days or later periods.
Split evidence
158 consecutive days of one loop detector split at random; deployment on that ramp sees new days, but neighbouring days of the same home stand sit on both sides R.
Label
as DodgerLoopDay. The day-of-week label was derived by the archive contributor from timestamps; the source study collected counts for unsupervised event detection R.
Open questions
Split: related periods (as DodgerLoopDay); whether this counts as clean is undecided.
EarthquakesSENSOR · Earth & space science · 322/139 train/test · length 512 · 2 classesfailstask replacementRelated periodsassumed new case: new periods or event groupstemporaltimeopen question
Taskfails
Source task
none Continuous seismic monitoring; no class target
Archive task
Whether a large event follows a window; curator thresholds
Label attached by
CuratorsredefinedS
Verdict
failstask replacement
The source collected the data but posed no classification target.
SplitRelated periods
New case assumed
New periods or event groups
Exact duplicates
none found D
Why
The time windows are described as non-overlapping, so shared time samples are not an established problem here. However, windows from one earthquake sequence can be assigned to both sets. They may share the same broader event. The split does not specifically test later periods or separately collected event groups.
Source
There is no external "original problem" here in the sense of a domain study that posed this task. The underlying data are hourly averaged seismic readings from the Northern California Earthquake Data Center (NCEDC), Dec 1967 to 2003, a monitoring archive rather than an experiment.
Publication
timeseriesclassification.com description page for Earthquakes, which is the curators' own account: "We transform this single time series into a classification problem by first defining a major event as any reading of over 5 on the Rictor scale... Hence we consider a positive case to be one where a major event is not preceded by another major event for at least 512 hours. To construct a negative case, we consider instances where there is a reading below 4... that is preceded by at least 20 readings in the previous 512 hours that are non-zero (to avoid trivial negative cases). None of the cases overlap in time (i.e. we perform a segmentation rather than use a sliding window). Of the 86,066 hourly readings, we produce 368 negative cases and 93 positive." No separate peer-reviewed paper describing this curation was located, nor documentation of exactly which NCEDC quantity was "averaged" per hour.
Full record
Benchmark task
Classify a 512-point hourly seismic window as preceding a major event or not.
Who would use it
seismic-hazard monitoring agencies and earthquake early-warning services, in principle; in practice the dataset was assembled as a TSC benchmark
Task construction
failtask replacement
Benchmark target: Whether a large event follows a window; curator thresholds. Source target or purpose: Continuous seismic monitoring; no class target. Label attached by: Curators. Mechanism 'task replacement': the archive replaces the source's label with a different task, or defines classes by its own rule where the source posed none.
Data origin
temporalindex: time
Archive changes
windowing
Leakage
pass (related periods)different periods of one ongoing process
Split sources
Curators' windows (non-overlapping 21-day windows of one 1967-2003 record) split at random by the curators RS
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails task construction
New case assumed
New periods or event groups.
Split evidence
the curators state the 512-hour cases are non-overlapping segments of one 1967-2003 NCEDC record S; how the windows were assigned to train and test is not documented, and windows of one seismic sequence can sit on both sides R.
Label
classes defined by the curators' thresholds and exclusion rules (a reading over 5 not preceded by another within 512 hours; negatives below 4 with at least 20 non-zero readings in the previous 512 hours), as stated on the archive page S.
Open questions
Related periods (windows of one seismic record); whether this counts as clean is undecided.
real task normal heartbeat vs supraventricular premature beat
Archive task
normal vs abnormal heartbeat (abnormal = supraventricular premature beat per the source thesis; the archive describes it as myocardial infarction)
Label attached by
Source studysource taskSR
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitConcern
New case assumed
New recording sessions of a known patient
Exact duplicates
none found D
Why
Each case is a heartbeat. The source thesis drew all 200 beats from one Holter recording of a single patient, so training and test beats share the patient and the recording session [S; that the shipped files are this collection is inferred from the attribution and matching 133:67 counts, R]. For patient-specific monitoring a new case is a new recording session of that patient, which the split cannot test.
Source
Cardiac diagnosis from electrocardiograms: distinguish normal heartbeats from supraventricular premature beats, single beats drawn from one Holter recording (SVDB) for Robert Olszewski's CMU thesis on feature extraction for structural pattern recognition. The archive describes the abnormal class as myocardial infarction, which the thesis contradicts; who converted the beats to 96 values is undocumented.
Publication
Olszewski, R., "Generalized feature extraction for structural pattern recognition in time-series data", PhD thesis, Carnegie Mellon University, 2001
Full record
Benchmark task
Classify a single-heartbeat series (length 96) as normal or abnormal (supraventricular premature beat per the source; the archive says MI).
Who would use it
cardiologists / automated ECG screening
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (normal heartbeat vs supraventricular premature beat; the archive misdescribes it as myocardial infarction; continuity into the shipped files inferred R); label origin: source task.
Data origin
temporalindex: time
Archive changes
windowingchannel-selectionz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Archive default split; who built it and by what rule is not documented RSR
Multichannel source
yes, one channel from a multichannel recording
One of two simultaneously recorded ECG leads (lead 0 / lead 1) is shipped; which one is undocumented.
Dataset family
none from the same source task
Overall verdict
fails leakage
New case assumed
new recording sessions of a known patient (author ruling).
Split evidence
the thesis draws all beats from one single-patient recording S; that the shipped files are this collection is inferred from the attribution and matching 133:67 counts R; the archive's 100/100 split and its time order are undocumented; +1 = normal is inferred R.
none none posed; a 20-hour research ECG of one severe congestive-heart-failure patient (PhysioBank chfdb record chf07)
Archive task
Five automatically assigned beat types
Label attached by
CuratorsredefinedSR
Verdict
failstask replacement
The source collected the data but posed no classification target.
SplitConcern
New case assumed
New patients or recording occasions
Exact duplicates
none found D
Why
The source record identified by the audit contains heartbeats from one patient during one long recording. Training and test beats therefore share that patient and recording. This can test more beats from that record, but it cannot establish performance on another patient or after the patient is recorded again.
Source
A 20-hour ECG of one patient with severe congestive heart failure (PhysioNet BIDMC chfdb record chf07), a research recording with no prediction problem attached; the UCR curators extracted single heartbeats and labelled them by automated annotation.
Publication
Archive page (timeseriesclassification.com), which cites Goldberger et al., "PhysioBank, PhysioToolkit, and PhysioNet", Circulation 101(23), 2000 for the recording and Hu, Chen & Keogh, "A general framework for never-ending learning from time series streams", Data Mining and Knowledge Discovery 29(6), 2015 for the first use. Neither primary paper was reached in full text for this record.
Full record
Benchmark task
Classify a single length-140 interpolated heartbeat into one of five automatically assigned beat types.
Who would use it
Cardiology / arrhythmia monitoring in principle; in practice the dataset was assembled as a benchmark for streaming time-series algorithms, not for a clinical decision.
Task construction
failtask replacement
Benchmark target: Five automatically assigned beat types. Source target or purpose: none posed; a 20-hour research ECG of one severe congestive-heart-failure patient (PhysioBank chfdb record chf07). Label attached by: Curators. Mechanism 'task replacement': the archive replaces the source's label with a different task, or defines classes by its own rule where the source posed none.
Data origin
temporalindex: time
Archive changes
windowingresamplingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Archive default split; who built it and by what rule is not documented RS
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails task construction and leakage
New case assumed
New patients or recording occasions.
Split evidence
all 5000 beats come from one patient's 20-hour recording (chfdb record chf07) S; a beat-type classifier is deployed on new patients R.
Label
the archive states the class values were obtained by automated annotation S; that the curators (Chen & Keogh) ran it, which annotator, and the five class names are not confirmed by any primary source R.
Open questions
Five class names are not confirmed by any primary source; 'derived' by automated annotation rather than replaced.
The source collected the data but posed no classification target.
SplitConcern
New case assumed
Independent recordings
Exact duplicates
none found D
Why
The label says which of two recording dates produced a heartbeat. Training and test beats come from those same two recordings of one person. The model may distinguish conditions specific to those recordings. The test does not show that the distinction holds when further independent recordings are collected.
Source
No original prediction problem is documented. The archive page says only "Data is from a 67 year old male" and that "The two classes are simply ECG date: 12/11/1990 / ECG date: 17/11/1990".
Publication
timeseriesclassification.com description page only (read 2026-08). No paper, thesis or database record was found; the supplied source link (www.pamap.org) is unrelated to ECG.
Full record
Benchmark task
Given a length-136 heartbeat from one 67-year-old man, decide whether it was recorded on 12 November 1990 or on 17 November 1990.
Who would use it
None. The recording is clinical in origin but nobody needs a model that predicts the calendar date of an ECG; the dataset exists purely as an algorithm benchmark.
Task construction
failmetadata as label
Benchmark target: Which of two dates, five days apart. Source target or purpose: None documented; one subject's clinical ECG. Label attached by: Curators. Mechanism 'metadata as label': the archive derives the classes from a recording attribute.
Data origin
temporalindex: time
Archive changes
windowingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
23 train / 861 test built by the curators; rule undocumented RS
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails task construction and leakage
New case assumed
Independent recordings.
Split evidence
the archive page states that all beats are from one 67-year-old male and that the classes are the two recording dates, so train and test beats come from the same two recordings of one subject S; even for a date-recognition task the test beats are slices of the recordings the model was trained on.
Label
recording-date label stated on the archive page S, attached by the curators R; source of the recording unknown. The archive's source link is unrelated to ECG; the date label and the single-subject claim rest on the archive sentence alone.
Open questions
Rests on the archive sentence alone; the archive's source link is unrelated.
real task device class of one separately metered household device (7 groups: kettle, immersion heater, washing machine, cold group, oven/cooker, screen group, dishwasher)
Archive task
appliance/device class (7 types)
Label attached by
Source studysource taskS
Verdict
real task
Lines et al. (IDEAL 2011) posed device recognition on the GEO trial data, to identify a new user's devices without previous labelled consumption; the authors grouped the ten common device types into seven classes, and SDM 2012 lists this exact 7-class daily task, so it is the source's own real task.
SplitConcern
New case assumed
New appliances or households
Exact duplicates
208 of 7,711 test series (2.7%) exactly match a training series D; 54 duplicate groups have conflicting labels
Why
The source kept each household in one set, but the shipped files differ in size from the published split, so it is unverified that the archive split keeps this. The value check finds 208 test cases identical to training cases, and 54 repeated-value groups carry conflicting labels. Most matches are two-valued on/off days, so they need not mean shared households.
Source
The Visual Energy Trial of the smart-meter company Green Energy Options (GEO) metered individual devices and total household electricity use in 187 homes across East Anglia. Readings were 15-minute consumption totals collected for about a year per household.
Publication
Lines et al., IDEAL 2011 (GEO Visual Energy Trial data); Bagnall et al., SDM 2012, Sec. 3.4; Lines PhD thesis (2015), Sec. 3.2.1; donated to the archive by Bagnall & Lines
Full record
Benchmark task
Classify a 96-point daily consumption profile into one of 7 device types.
Who would use it
GEO and its smart-meter users; automatic device recognition for consumption feedback
Task construction
passnone (source task label)
The benchmark label is the source study's own 7-group device class (ten device types, with cold and screen devices merged by the source authors, thesis p.40); label origin: source task, attached by the source study S.
Data origin
temporalindex: time
Archive changes
windowingz-normalization
Leakage
failexact cross-split matches (208 of 7711 test series, 54 duplicate groups with conflicting labels)
Split sources
Source split kept households apart (thesis p.40; IDEAL 2011 Sec. 4) S; the shipped sizes (8926/7711) differ from SDM 2012's 8953/7745, so whether the archive split keeps this is unverified RD
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails leakage
New case assumed
new households. The Concern verdict comes from the 208 exact matches D; 207 of them are two-valued on/off days, so they need not mean shared households R. The source split was household-exclusive S, but its correspondence to the shipped files is unverified. 54 duplicate groups carry conflicting labels D.
Open questions
Whether the shipped split (8926/7711) keeps households apart as the source split (8953/7745) did; which numeric label (1-7) is which device group; who z-normalized the data, and when.
EOGHorizontalSignalEOG · Human motion & gesture · 362/362 train/test · length 1250 · 12 classesreal tasksource task labelUnclearassumed new case: new people or recording occasionstemporaltimesingle channelfamily: eogopen question
Taskreal task
Source task
real task which of 12 Japanese Katakana stroke types was eye-written
Archive task
which of 12 Japanese Katakana stroke types was eye-written
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New people or recording occasions
Exact duplicates
none found D
Why
The eye-movement samples come from six people. The audit describes splitting individual movements rather than keeping all movements from a person together. We cannot establish performance on unfamiliar users. Sharing users can suit a personalised system, but its test should explain whether they were measured again on a separate occasion.
Source
Fang & Shinozaki (2018) built an electrooculography eye-writing interface for ALS patients who can still move their eyes; each series is the horizontal channel of one isolated Japanese Katakana stroke from six participants.
Publication
Fang, Fuming, and Takahiro Shinozaki, "Electrooculography-based continuous eye-writing recognition system for efficient assistive communication systems", PLoS ONE 13(2): e0192684, 2018. Read 2026-08 via the PLOS article page. Methods: "A two-channel (horizontal and vertical) EOG signal was obtained by taking the differences of the signals between the electrodes on the left and right sides and those between the upper and lower sides, respectively"; the amplifier is a "commercial biomedical amplifier, BlueGain (Cambridge Research Systems Ltd.)"; six healthy participants (one female, five male), aged 21-31, one of whom was an eye-writing expert; Data recording > Isolated data: "In general, each of the 12 types of strokes was recorded 10 times for each participant, but a few strokes were recorded nine or 11 times. In total, 724 strokes were recorded from all the participants." Setup for isolated eye-writing recognition: "We used 10-fold cross-validation for performance evaluation" and "Both the DTW template and HMM were user-dependent" (how the folds were formed is not stated).
Full record
Benchmark task
Classify a length-1250 single-channel EOG series (horizontal (left-right) eye movement) into one of 12 Katakana stroke types. 362 train / 362 test.
Who would use it
People with severe motor impairment (ALS, locked-in syndrome) and the clinicians /engineers building assistive communication devices for them.
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of 12 Japanese Katakana stroke types was eye-written); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded two-channel electrooculography recorded around the left eye; the UCR release keeps a single channel of it. The signal is intrinsically two-dimensional - eye-writing a stroke is a planar movement, and the horizontal and vertical channels are the two coordinates of it. This dataset keeps only the horizontal (left-right) component, so strokes that differ mainly along the other axis become indistinguishable in principle. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
the horizontal and vertical channels of the same recordings, identical split
Overall verdict
real task; leakage unknown
New case assumed
New people or recording occasions.
Split evidence
six participants recorded 724 isolated strokes in total (Fang & Shinozaki 2018, "Data recording"), and the archive's 362 + 362 = 724, so train and test together hold all the isolated strokes S. How the strokes were divided between train and test, and whether the same participants appear on both sides, is not documented; the archive ships no participant identifier R. The source's own isolated-stroke models were user-dependent, evaluated by 10-fold cross-validation S; an assistive interface used by a new person would need a participant-disjoint split R. Hence Unclear R.
Open questions
How the 724 strokes were divided between train and test, and whether participants are shared, is not documented; matching the archive series against the source's S1 Database (isolated data) could settle it.
EOGVerticalSignalEOG · Human motion & gesture · 362/362 train/test · length 1250 · 12 classesreal tasksource task labelUnclearassumed new case: new people or sessionstemporaltimesingle channelfamily: eogopen question
Taskreal task
Source task
real task which of 12 Japanese Katakana stroke types was eye-written
Archive task
which of 12 Japanese Katakana stroke types was eye-written
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New people or sessions
Exact duplicates
none found D
Why
Same as EOGHorizontalSignal. [EOGHorizontalSignal: The eye-movement samples come from six people. The audit describes splitting individual movements rather than keeping all movements from a person together. We cannot establish performance on unfamiliar users. Sharing users can suit a personalised system, but its test should explain whether they were measured again on a separate occasion.]
Source
Fang & Shinozaki (2018) built an electrooculography eye-writing interface for ALS patients who can still move their eyes; each series is the vertical channel of one isolated Japanese Katakana stroke from six participants.
Publication
Fang, Fuming, and Takahiro Shinozaki, "Electrooculography-based continuous eye-writing recognition system for efficient assistive communication systems", PLoS ONE 13(2): e0192684, 2018. Read 2026-08 via the PLOS article page. Methods: "A two-channel (horizontal and vertical) EOG signal was obtained by taking the differences of the signals between the electrodes on the left and right sides and those between the upper and lower sides, respectively"; the amplifier is a "commercial biomedical amplifier, BlueGain (Cambridge Research Systems Ltd.)"; six healthy participants (one female, five male), aged 21-31, one of whom was an eye-writing expert; Data recording > Isolated data: "In general, each of the 12 types of strokes was recorded 10 times for each participant, but a few strokes were recorded nine or 11 times. In total, 724 strokes were recorded from all the participants." Setup for isolated eye-writing recognition: "We used 10-fold cross-validation for performance evaluation" and "Both the DTW template and HMM were user-dependent" (how the folds were formed is not stated).
Full record
Benchmark task
Classify a length-1250 single-channel EOG series (vertical (up-down) eye movement) into one of 12 Katakana stroke types. 362 train / 362 test.
Who would use it
People with severe motor impairment (ALS, locked-in syndrome) and the clinicians /engineers building assistive communication devices for them.
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of 12 Japanese Katakana stroke types was eye-written); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded two-channel electrooculography recorded around the left eye; the UCR release keeps a single channel of it. The signal is intrinsically two-dimensional - eye-writing a stroke is a planar movement, and the horizontal and vertical channels are the two coordinates of it. This dataset keeps only the vertical (up-down) component, so strokes that differ mainly along the other axis become indistinguishable in principle. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
the horizontal and vertical channels of the same recordings, identical split
Overall verdict
real task; leakage unknown
New case assumed
New people or sessions.
Split evidence
six participants recorded 724 isolated strokes in total (Fang & Shinozaki 2018, "Data recording"), and the archive's 362 + 362 = 724, so train and test together hold all the isolated strokes S. How the strokes were divided between train and test, and whether the same participants appear on both sides, is not documented; the archive ships no participant identifier R. The source's own isolated-stroke models were user-dependent, evaluated by 10-fold cross-validation S; an assistive interface used by a new person would need a participant-disjoint split R. Hence Unclear R.
Open questions
How the 724 strokes were divided between train and test, and whether participants are shared, is not documented; matching the archive series against the source's S1 Database (isolated data) could settle it.
EthanolLevelSPECTRO · Spectroscopy & chemistry · 504/500 train/test · length 1751 · 4 classesreal tasksource task labelSeparatedassumed new case: new bottle typesordered, non-temporalwavelength or wavenumberfamily: ethanol
Taskreal task
Source task
real task ethanol concentration class of the liquid in the bottle (35%, 38%, 40%, 45%)
Archive task
ethanol concentration class of the liquid in the bottle (35%, 38%, 40%, 45%)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitSeparated
New case assumed
New bottle types
Exact duplicates
none found D
Why
The archive paper says that a bottle type is never used in both training and testing (Dau et al. 2019). The test therefore checks measurements involving unfamiliar bottle types. This is a documented source separation. It does not, by itself, establish that every other recording condition is different between the sets.
Source
Spirit-fraud screening, from a University of East Anglia project with the Scotch Whisky Research Institute: counterfeit spirits are typically bottled with the wrong alcohol strength, so measuring ethanol content through the sealed bottle would let an inspector screen a suspect batch without opening it.
Publication
Dau, Bagnall, Kamgar, Yeh, Zhu, Gharghabi, Ratanamahatana, Keogh, "The UCR Time Series Archive", arXiv:1810.07758, Sec. V - read (PDF). The two primary references the archive gives - Lines, Taylor, Bagnall (ICDM 2016), which introduced the dataset, and Large, Kemsley, Wellner, Goodall, Bagnall (PAKDD 2018) - were not reached; the PAKDD paper is paywalled at Springer. The closely related UEA dataset EthanolConcentration is documented first-hand by its donor in Bagnall et al. (arXiv:1811.00075) Sec. 7.1, read in full, and describes the same instrument setup.
Full record
Benchmark task
Classify a univariate length-1751 through-the-bottle spectrum into one of four ethanol concentrations.
Who would use it
trading-standards / customs inspectors and the drinks industry, screening batches of suspect spirits non-invasively
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (ethanol concentration class of the liquid in the bottle (35%, 38%, 40%, 45%)); label origin: source task.
Data origin
ordered, non-temporalindex: wavelength or wavenumber
Archive changes
spectrum-as-series
Leakage
passsource separation documented
Split sources
Dau et al. 2019 (p. 7, p. 10): the same bottle never appears in both train and test SS
Multichannel source
no
Dataset family
ethanol with EthanolConcentrationUEA
the same through-the-bottle spectroscopy programme; UEA EthanolConcentration is the later multivariate release
Overall verdict
real task with a documented split
New case assumed
New bottle types.
Split evidence
Dau et al. 2019 p. 7 and p. 10: the same bottle never appears in both train and test S.
toy task Head profiles recorded at UCR; no applied problem
Archive task
Which of 14 graduate students, from head profile
Label attached by
Collectors (= curators)instance identityS
Verdict
failstoy source task · instance identity as label (primary: instance identity as label)
The source data were staged or generated to demonstrate a method.
SplitConcern
New case assumed
New photography occasions
Exact duplicates
1 of 1,690 test series (0.1%) exactly match a training series D
Why
The task identifies people, so those identities are expected in both sets. The concern is whether test photographs were taken on a different occasion. The audit does not establish this. Similar photos taken under the same conditions may make identification easier than recognising the person in newly collected photographs. The value check also reports one test case identical to training data.
Source
A person-identification problem built in-house at UCR. Archive page: "FaceAll was generated at UCR by Xiaopeng Xi.
Publication
Archive page only (donated by X. Xi and E. Keogh; no data_source_link or second_link given). Xi et al., "Fast time series classification using numerosity reduction", ICML 2006 was read and does not describe this dataset (it uses "Face (four)", n=112). No primary description was located.
Full record
Benchmark task
Classify a length-131 head-profile series as one of 14 individuals.
Who would use it
none applied; biometric identification from a head silhouette is not a deployed use case, and the data were made as a research benchmark
Task construction
failtoy source task · instance identity as label (primary: instance identity as label)
Benchmark target: Which of 14 graduate students, from head profile. Source target or purpose: Head profiles recorded at UCR; no applied problem. Label attached by: Collectors (= curators). Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
failexact cross-split matches (1 of 1690 test series)
Split sources
Curators' split 560/1690; FacesUCR re-splits the same 2250 instances as 200/2050 after rotation alignment; rules undocumented RDR
the same 2250 head-profile instances with a different alignment and split
Overall verdict
fails task construction and leakage
New case assumed
New photography occasions.
Split evidence
only 14 subjects with many images each; images of the same person in train and test are near-duplicates, so the test does not present a new photograph of a known person R. Head profiles of 14 graduate students recorded at UCR; no applied problem behind the collection (archive page only) R.
FaceFourIMAGE · Shapes & object images · 24/88 train/test · length 350 · 4 classeslabel meaning undocumentedcounted as a real taskUnclearassumed new case: new source images or sessionsordered, non-temporalcontour arc-lengthopen question
Tasklabel meaning undocumented
Source task
undocumented which of four face classes (presumably individuals); provenance undocumented
Archive task
which of four face classes (presumably individuals)
Label attached by
UnknownundocumentedR
Verdict
label meaning undocumented
No source describes who collected the faces or why, and the archive disclaims the provenance, so the original task is undocumented (counted with the real tasks).
SplitUnclear
New case assumed
New source images or sessions
Exact duplicates
none found D
Why
The available record does not adequately describe the people, source photographs or capture sessions. We cannot check whether related images or acquisitions occur in both sets. No specific overlap is established by this absence of documentation.
Source
A four-class face-outline problem of unclear origin: the archive page says 'we are not sure of the exact provenance'; one of the oldest datasets in the archive (Ratanamahatana & Keogh).
Publication
Archive page only, which explicitly disclaims knowledge of provenance. Xi et al., "Fast time series classification using numerosity reduction", ICML 2006 (read) lists "Face (four)" with 112 instances but gives no description of the data.
Full record
Benchmark task
Classify a length-350 face-outline series into one of four classes.
Who would use it
none identified; used as a benchmark for time-series similarity measures
Task construction
passnone (label meaning undocumented)
Benchmark target: which of four face classes (presumably individuals). No source documents what the classes mean or who attached them; label origin: undocumented (counted with the real tasks).
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
label meaning undocumented (counted with the real tasks); leakage unknown
New case assumed
New source images or sessions.
Split evidence
nothing about subjects or collection is documented R. The archive page disclaims knowledge of the source; label meaning undocumented, counted with the real tasks R.
Open questions
Provenance undocumented; label meaning undocumented (counted with the real tasks).
toy task Head profiles recorded at UCR; no applied problem
Archive task
Which of 14 graduate students, from head profile (rotation-aligned, different split)
Label attached by
Collectors (= curators)instance identityS
Verdict
failstoy source task · instance identity as label (primary: instance identity as label)
The source data were staged or generated to demonstrate a method.
SplitUnclear
New case assumed
New photography occasions
Exact duplicates
none found D
Why
The task uses reprocessed photographs from the same collection as FaceAll. Identifying familiar people is expected, but the audit does not establish that test photos were taken on a separate occasion. Similar lighting, pose or camera conditions could help recognition without showing that it works in newly collected photographs.
Source
The same 14-subject head-profile collection as FaceAll, re-processed. Archive page: "FacesAll is rotationally aligned version of FaceAll with a different train/test split"; FaceAll's page records that it "was generated at UCR by Xiaopeng Xi.
Publication
Archive page only (donated by X. Xi and E. Keogh; no data_source_link or second_link). No primary publication describing the collection was located.
Full record
Benchmark task
Classify a length-131 rotationally aligned head-profile series as one of 14 individuals.
Who would use it
none applied; created as a research benchmark
Task construction
failtoy source task · instance identity as label (primary: instance identity as label)
Benchmark target: Which of 14 graduate students, from head profile (rotation-aligned, different split). Source target or purpose: Head profiles recorded at UCR; no applied problem. Label attached by: Collectors (= curators). Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Curators' split 200/2050 of the same 2250 instances as FaceAll after rotation alignment; rule undocumented RR
the same 2250 head-profile instances with a different alignment and split
Overall verdict
fails task construction (leakage unknown)
New case assumed
New photography occasions.
Split evidence
same 2250 instances as FaceAll, re-aligned and re-split; near-duplicate images of the same person span the split R. The 14 identities are on both sides by construction of an identification task; without an exact match or documented occasion sharing the verdict is Unclear (FaceAll is Concern only because of its one exact match) R.
Open questions
Same identities on both sides by construction; Unclear because there is no exact match and occasion sharing is undocumented, whereas FaceAll is Concern through its one exact match.
real task word spotting: matching handwritten word images by word identity to index historical manuscripts
Archive task
which word a word-image is an instance of (50 word types)
Label attached by
CuratorsredefinedSR
Verdict
failstask replacement
The source study posed word-image matching (word spotting) to index handwritten manuscripts, a task libraries and archives need.
SplitUnclear
New case assumed
New manuscript pages
Exact duplicates
none found D
Why
The word images come from one writer's manuscripts. Words on the same page can share handwriting and scanning conditions. We do not know whether whole pages were kept in one set. The test may therefore contain more words from familiar pages, rather than words from pages the model has never seen.
Source
Word spotting in George Washington's handwritten manuscripts (Rath & Manmatha, CVPR 2003): word images are matched by column profiles so that historical collections can be indexed without transcription.
Publication
Rath & Manmatha, "Word image matching using dynamic time warping", CVPR 2003 (read via the UMass CIIR copy linked from the archive page).
Full record
Benchmark task
Classify a length-270 word-profile series into one of 50 word classes.
Who would use it
libraries and archives wanting searchable indexes over handwritten collections that are too large to transcribe by hand
Task construction
failtask replacement
Benchmark target: which word a word-image is an instance of (50 word types), a closed set of 50 words carved out of the source's retrieval pool. Source target or purpose: word spotting, matching word images by their ASCII tags to index the George Washington manuscripts (Rath & Manmatha 2003, Sec. 3.1-3.2). Label attached by: Curators. Mechanism 'task replacement': the archive replaces the source's label with a different task, or defines classes by its own rule where the source posed none; here 50 word classes were selected from about 2,400 word images that the source used only for retrieval, and who chose them is undocumented.
Data origin
ordered, non-temporalindex: spatial position
Index is horizontal position across the word image (one value per image column), not time.
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Curators' split 450/455; WordSynonyms re-splits the same 905 instances as 267/638; rules undocumented RR
the same 905 word images, mirrored and re-split; WordSynonyms merges the 50 classes into 25
Overall verdict
fails task construction (leakage unknown)
New case assumed
New manuscript pages.
Split evidence
word images from one writer's manuscripts; deployment indexes new pages of the same collection; page-level split undocumented R.
Label
the word identity is the source's own label; the change is retrieval re-posed as closed-set classification (flag re-posed), not a label replacement R. The source's word-retrieval task is re-posed as closed-set 50-class classification; the word identity is the source's own label R.
Open questions
Word retrieval re-posed as closed-set classification; not counted among the constructed tasks.
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New physical specimens
Exact duplicates
none found D
Why
The cases are outlines derived from fish images. The description does not establish how physical specimens and any repeated views were assigned to the two sets. New-specimen evaluation therefore cannot be verified; repeated views are a possibility to check, not an observed overlap.
Source
FishID (Lee et al. 2008), a machine-vision system for counting migrating fish by species at a dam fishway; the archive series are unrolled fish outlines derived at UCR from that work's images.
Publication
Lee, Archibald, Schoenberger, Dennis & Shiozawa, "Contour Matching for Fish Species Recognition and Migration Monitoring", in Applications of Computational Intelligence in Biology, Springer, 2008, pp. 183-207 - only pp. 183-184 (Springer preview) were read; the rest is paywalled. Summary: "we present results from a field test of the system at Prosser Dam in Prosser, Washington. In tests with up to four distinct species, the algorithm correctly determines the species with greater than 90 percent accuracy." The data donors' study is the earlier SPIE version: Lee, Schoenberger, Shiozawa, Xu & Zhan, "Contour Matching for a Fish Recognition and Migration Monitoring System", SPIE Optics East 2004, vol. 5606 (author copy read in full, 2026-11-02), Sec. 5.1: "Seven fish species that have similar shape characters were chosen for study. They are Chinook Salmon (SA), Winter Coho (WC), brown trout (BT), Bonneville cutthroat (BC), Colorado River cutthroat trout (CRC), Yellowstone Cutthroat (YC) and mountain whitefish (WF). We used a total of 64 images to generate shapes of these seven species"; Sec. 5.2: "Each contour was used to compare against other contours in the data set" (best accuracy 64%). Keogh et al. (VLDB 2006, expanded version) list "Fish 7 [classes] 350 [instances]" and write: "The data donors of the fish dataset tested many shape descriptors ... to achieve the best error rate of 36.0% [19]", where [19] is the SPIE 2004 paper.
Full record
Benchmark task
Classify a length-463 fish-outline series into one of seven species.
Who would use it
fisheries agencies and dam operators counting migrating fish by species
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (fish species (7 classes)); label origin: source task.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New physical specimens.
Split evidence
outline extraction at UCR undocumented, so how fish and images map to series and to the split is unknown R.
FordASENSOR · Industrial, sensors & robotics · 3,601/1,320 train/test · length 500 · 2 classesreal tasksource task labelUnclearassumed new case: new vehicles or recording occasionstemporaltime
Taskreal task
Source task
real task whether a certain fault symptom exists in the automotive subsystem (+1 exists / -1 does not)
Archive task
whether a certain fault symptom exists in the automotive subsystem (+1 exists / -1 does not)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New vehicles or recording occasions
Exact duplicates
none found D
Why
The files do not identify the vehicle, test rig or recording occasion behind each case. We therefore cannot check whether these sources were kept apart. The split may be appropriate, but the available information does not show that testing uses vehicles or recording occasions absent from training.
Source
An industrial classification challenge posed by Ford for the IEEE World Congress on Computational Intelligence 2008.
Publication
"Ford Classification Challenge", the competition's own website (home.comcast.net/~nn_classification/, dead since ~2016; read via the Internet Archive snapshot of 2008-07-06). Challenge section: "Data samples from an automotive subsystem were collected in batches of 500 samples per diagnostic session. The objective is a classifier that will determine whether a certain symptom exists or does not exist after examining the samples." Facts listed there: "The 500 samples collected in each diagnostic session represent a set of sequential values of the measured variable, where sample n+1 occurs after sample n." and "The beginning of the sampling process is not aligned with any external circumstance or any aspect of the observed pattern." On this dataset specifically: "Ford_A: Data samples of known and hidden classification were collected in typical operating conditions, with minimal noise contamination." The page's table gives Ford_A: 3271 training, 330 validation, 1320 test patterns — 3271 + 330 = 3601, exactly the archive's n_train, so UCR merged the competition's train and validation sets and kept its test set.
Full record
Benchmark task
Classify a univariate length-500 engine-diagnostic series as symptom-present or symptom-absent.
Who would use it
automotive manufacturers and service workshops performing automated end-of-line or in-service fault diagnosis
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (whether a certain fault symptom exists in the automotive subsystem (+1 exists / -1 does not)); label origin: source task.
Data origin
temporalindex: time
Archive changes
z-normalization
Leakage
unknownsource separation undocumented
Split sources
Ford Classification Challenge split (competition organisers); vehicle and rig identities withheld SR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New vehicles or recording occasions.
Split evidence
competition data: which vehicle or rig each batch came from is withheld, so session-level grouping cannot be checked R.
FordBSENSOR · Industrial, sensors & robotics · 3,636/810 train/test · length 500 · 2 classesreal tasksource task labelUnclearassumed new case: new conditions and recordingstemporaltime
Taskreal task
Source task
real task whether a certain fault symptom exists in the automotive subsystem (+1 exists / -1 does not)
Archive task
whether a certain fault symptom exists in the automotive subsystem (+1 exists / -1 does not)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New conditions and recordings
Exact duplicates
none found D
Why
The audit describes test data collected under different, noisier operating conditions. This tests a change in conditions; it does not demonstrate shared recordings. We still do not know which vehicles or recording occasions produced the cases, so we cannot check whether those sources are different between the sets.
Source
The second half of Ford's IEEE WCCI 2008 classification challenge: detect a fault symptom in an automotive subsystem from a 500-sample diagnostic batch, with the test data deliberately recorded under noisier operating conditions than the training data.
Publication
"Ford Classification Challenge", the competition's own website (home.comcast.net/~nn_classification/, dead since ~2016; read via the Internet Archive snapshot of 2008-07-06). Challenge section: "Data samples from an automotive subsystem were collected in batches of 500 samples per diagnostic session. The objective is a classifier that will determine whether a certain symptom exists or does not exist after examining the samples." On this dataset specifically: "Ford_B: Data samples of known classification were collected in typical operating conditions, while data samples of hidden classification were collected under noisy conditions." The page also warns: "Note: the two data sets are not related to each other and should not be mixed." Its table gives Ford_B: 3306 training, 330 validation, 810 test patterns — 3306 + 330 = 3636, exactly the archive's n_train, so UCR merged the competition's train and validation sets and kept its test set.
Full record
Benchmark task
Classify a univariate length-500 engine-diagnostic series as symptom-present or symptom-absent, where the test set was recorded under noisier operating conditions than the training set.
Who would use it
automotive manufacturers and service workshops needing diagnosis that survives noisy real-world operating conditions
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (whether a certain fault symptom exists in the automotive subsystem (+1 exists / -1 does not)); label origin: source task.
Data origin
temporalindex: time
Archive changes
z-normalization
Leakage
unknownsource separation undocumented
Split sources
Ford Classification Challenge split; the test set was recorded under noisier operating conditions than the training set SSR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New conditions and recordings.
Split evidence
the test set was recorded under different, noisier operating conditions than the training set (competition design), which is a documented condition shift S; vehicle-level grouping is still withheld R.
The source collected the data but posed no classification target.
SplitConcern
New case assumed
New physical freezers
Exact duplicates
1 of 2,850 test series (0%) exactly match a training series D
Why
The source contains ongoing measurements of two freezers in one house. Pieces of the same recording can share appliance settings and operating conditions. The audit does not show that testing was kept for a later period or a new recording occasion. The value check separately reports 1 of 2,850 test cases matching training data.
Source
The data come from the REFIT electrical load measurements study, which continuously monitored whole-house aggregate load plus nine individual appliances in each of 20 UK households at 8-second intervals over two years.
Publication
REFIT project ("Personalised Retrofit Decision Support Tools for UK Homes using Smart Home Technology"); primary data descriptor read: Murray, Stankovic & Stankovic, "An electrical load measurements dataset of United Kingdom households from a two-year longitudinal study", Scientific Data 4, 160122 (2017), Abstract: "The REFIT electrical load measurements dataset described in this paper includes whole house aggregate loads and nine individual appliance measurements at 8-second intervals per house, collected continuously over a period of two years from 20 houses." Curation described in Dau et al., "The UCR Time Series Archive", arXiv:1810.07758, Sec. VI-B-5 ("Electrical Load Measurement data - Freezer data sets"): "All data are from freezers in House 1. This data set has two classes, one representing the power demand of the fridge freezer in the kitchen, the other representing the power demand of the (less frequently used) freezer in the garage."
Full record
Benchmark task
Classify a 301-point power-demand window as coming from the kitchen fridge-freezer or the garage freezer of REFIT House 1.
Who would use it
REFIT's stakeholders are energy-feedback and retrofit-advice services; the specific "which of these two freezers" question has no operational user and was constructed as an algorithm benchmark (a hard, locally-discriminable pair).
Task construction
failinstance identity as label
Benchmark target: Which of two freezers in one house. Source target or purpose: REFIT load monitoring; no class target. Label attached by: Curators. Mechanism 'instance identity as label': the class is which of two physical appliances produced the window.
Data origin
temporalindex: time
Archive changes
windowingz-normalization
Leakage
failexact cross-split matches (1 of 2850 test series) + documented or data-verified shared source
Split sources
Dau et al. 2019 (Sec. VI-B-5): one 2850-series test set shared with FreezerSmallTrain; 150 training series SSDR
the same REFIT House 1 data, identical test set, two training-set sizes
Overall verdict
fails task construction and leakage
New case assumed
new physical freezers (the card: 'nothing shows the result would transfer to other freezers') R.
Split evidence
Dau et al. Sec. VI-B-5 states all data are from the two House 1 freezers, one per class, so the same two appliances are on both sides S; windows of two continuous recordings, so train and test windows can be minutes apart R; 1 of 2,850 test windows is identical to a training window D.
Label
Dau et al. Sec. VI-B-5: the UCR curators built the two-freezer task from REFIT House 1 channels; REFIT posed no classification task S. The class is which of two sub-meter channels (kitchen fridge-freezer vs garage freezer) of REFIT House 1 a window came from; REFIT posed no classification task S.
Which of two freezers in one house (smaller training split)
Label attached by
Curatorsinstance identityS
Verdict
failsinstance identity as label
The source collected the data but posed no classification target.
SplitConcern
New case assumed
New physical freezers
Exact duplicates
27 of 2,850 test series (0.9%) exactly match a training series D
Why
Same as FreezerRegularTrain. This smaller-training-set variant has 27 of 2,850 test cases matching training data in the value check. [FreezerRegularTrain: The source contains ongoing measurements of two freezers in one house. Pieces of the same recording can share appliance settings and operating conditions. The audit does not show that testing was kept for a later period or a new recording occasion. The value check separately reports 1 of 2,850 test cases matching training data.]
Source
The data come from the REFIT electrical load measurements study, which continuously monitored whole-house aggregate load plus nine individual appliances in each of 20 UK households at 8-second intervals over two years.
Publication
REFIT project ("Personalised Retrofit Decision Support Tools for UK Homes using Smart Home Technology"); primary data descriptor read: Murray, Stankovic & Stankovic, "An electrical load measurements dataset of United Kingdom households from a two-year longitudinal study", Scientific Data 4, 160122 (2017), Abstract: "The REFIT electrical load measurements dataset described in this paper includes whole house aggregate loads and nine individual appliance measurements at 8-second intervals per house, collected continuously over a period of two years from 20 houses." Curation described in Dau et al., "The UCR Time Series Archive", arXiv:1810.07758, Sec. VI-B-5 ("Electrical Load Measurement data - Freezer data sets"): "All data are from freezers in House 1. This data set has two classes, one representing the power demand of the fridge freezer in the kitchen, the other representing the power demand of the (less frequently used) freezer in the garage."
Full record
Benchmark task
Classify a 301-point power-demand window as coming from the kitchen fridge-freezer or the garage freezer of REFIT House 1, training on only 28 labelled examples (the small-train variant, meant for data-scarce benchmarking).
Who would use it
REFIT's stakeholders are energy-feedback and retrofit-advice services; the specific "which of these two freezers" question has no operational user and was constructed as an algorithm benchmark (a hard, locally-discriminable pair).
Task construction
failinstance identity as label
Benchmark target: Which of two freezers in one house (smaller training split). Source target or purpose: REFIT load monitoring; no class target. Label attached by: Curators. Mechanism 'instance identity as label': the class is which of two physical appliances produced the window.
Data origin
temporalindex: time
Archive changes
windowingz-normalization
Leakage
failexact cross-split matches (27 of 2850 test series) + documented or data-verified shared source
Split sources
Dau et al. 2019 (Sec. VI-B-5): the same 2850-series test set as FreezerRegularTrain; 28 training series SSDR
the same REFIT House 1 data, identical test set, two training-set sizes
Overall verdict
fails task construction and leakage
New case assumed
new physical freezers (the card: 'nothing shows the result would transfer to other freezers') R.
Split evidence
Dau et al. Sec. VI-B-5 states all data are from the two House 1 freezers, one per class, so the same two appliances are on both sides S; windows of two continuous recordings, so train and test windows can be minutes apart R; 27 of 2,850 test windows are identical to training windows D.
Label
as FreezerRegularTrain. The class is which of two sub-meter channels (kitchen fridge-freezer vs garage freezer) of REFIT House 1 a window came from; REFIT posed no classification task S.
real task fungal species (18 species; Candida, Cryptococcus, Aspergillus and related genera)
Archive task
fungal species (18 species; Candida, Cryptococcus, Aspergillus and related genera)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitConcern
New case assumed
New biological strains
Exact duplicates
none found D
Why
The source measured fungal strains repeatedly. The audit reports that testing includes another measurement of a strain used in training. A model could recognise that familiar strain. This is a narrower test than identifying the species of a newly collected strain, for which all measurements of a strain should be kept together.
Source
A clinical-microbiology diagnostics problem - fungal infections need rapid, sensitive species identification, and high-resolution melt (HRM) analysis of a PCR-amplified DNA region is a cheap molecular assay whose curve shape is species-specific.
Publication
Lu, Mirchevska, Phatak, Li, Luka, Calderone, Fonzi, "Dynamic time warping assessment of high-resolution melt curves provides a robust metric for fungal identification", PLoS ONE 12(3), e0173320, 2017 (open access, read). Methods, "PCR amplification" - "Samples were then heated from 60-95 C at a r[ate] ... [with] acquisition of 25 fluorescent readings per 1 C. All samples were tested in duplicate in two independent trials." Methods, "High-resolution melt curve analysis" - "melting rates were directly determined using the first derivative of the normalized melt curves. Melt curve data was normalized within the LightCycler 480 Gene Scanning Software (Roche). The region between 72 and 74 C was set at 100%, 93.5 to 94 C was set to 0%. No curve shifting was applied. ... Rate curves were fitted with a cubic smoothing spline ... and the splines were used to calculate rates at 0.1 C increments. The resulting curves were z-normalized using the scale function and the region between 76 and 94 C was used for distance calculations." Introduction - "The sequentially ordered data of a melt curve is comparable to the temporally ordered data of a time series", i.e. the authors themselves frame the time-series treatment as an analogy.
Full record
Benchmark task
Classify a univariate length-201 melt-rate curve into one of 18 fungal species.
Who would use it
clinical microbiology laboratories identifying the fungal species causing an infection
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (fungal species (18 species; Candida, Cryptococcus, Aspergillus and related genera)); label origin: source task.
Data origin
ordered, non-temporalindex: temperature (melt curve)
Index is temperature along a DNA melt curve (monotone in time); filed as ordered, non-temporal because the shipped index is temperature, not time.
Archive changes
resamplingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Archive default split; who built it and by what rule is not documented RSR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails leakage
New case assumed
New biological strains.
Split evidence
51 strains each run in duplicate in two trials; with one training curve per species, the replicates of every training strain are necessarily in the test set; a clinical lab identifies new isolates R. The index is temperature of a high-resolution melt curve (monotone in time); filed as ordered, non-temporal R.
Open questions
Index is temperature along a DNA melt curve (monotone in time); filed as ordered, non-temporal because the shipped index is temperature, not time; a reader who regards a melt curve, monotone in time, as a time axis would count it as temporal.
GestureMidAirD1HAR · Human motion & gesture · 208/130 train/test · length variable · 26 classesreal tasksource task labelSeparatedassumed new case: new peopletemporaltimesingle channelfamily: gesture-midair
Taskreal task
Source task
real task which of 26 mid-air interface-command gestures was drawn
Archive task
which of 26 mid-air interface-command gestures was drawn
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitSeparated
New case assumed
New people
Exact duplicates
none found D
Why
The archive description says that no person occurs in both sets (archive dataset description). The test therefore uses people absent from training. There is disagreement in the records about which participant group is training, but both describe keeping the groups separate. No shared-person concern is identified by that description.
Source
A human-computer-interaction study of mid-air command gestures: 13 people drew each of 26 stroke-like commands (check, delete, circle, star, swipes, brackets, 3D polylines, ...) in the air, and a Leap Motion controller tracked the dominant-hand forefinger, giving a 3D trajectory per gesture.
Publication
Caputo, Prebianca, Carcangiu, Spano, Giachetti, "Comparing 3D trajectories for simple mid-air gesture recognition", Computers & Graphics 73 (2018) 17-25. Paper not read (paywalled, ScienceDirect returns HTTP 403). Layer 1 rests on the archive description page plus the authors' data/code repository https://github.com/giach68/gesturesCodes, which was inspected directly.
Full record
Benchmark task
Classify a variable-length series of one spatial coordinate of a mid-air fingertip trajectory into 26 gesture classes, training on 8 subjects and testing on 5 disjoint subjects.
Who would use it
designers of mid-air gestural interfaces (touchless control of displays, VR/AR applications)
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of 26 mid-air interface-command gestures was drawn); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selection
Leakage
passsource separation documented
Split sources
Archive page: 'Data of a same subjects does not appear in both train and test set' (8 subjects train, 5 test; the source repository lists the reverse assignment) SRS
Multichannel source
yes, one channel from a multichannel recording
The source recorded 3D position trajectory of the dominant-hand forefinger captured by a Leap Motion controller; the UCR release keeps a single channel of it. Two of the three spatial coordinates are discarded, which is severe for a stroke-shape task: several of the 26 gestures (e.g. the six poly3D variants) differ only in the ordering of movements across axes. The explicit timestamp column of the raw data is dropped, so the index is treated as uniform time. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
D1, D2 and D3 are the x, y and z coordinates of the same fingertip trajectories
Overall verdict
real task with a documented split
New case assumed
New people R.
Split evidence
archive page: 'Data of a same subjects does not appear in both train and test set' (8 subjects train, 5 test) S; the repository lists the reverse assignment R.
GestureMidAirD2HAR · Human motion & gesture · 208/130 train/test · length variable · 26 classesreal tasksource task labelSeparatedassumed new case: new peopletemporaltimesingle channelfamily: gesture-midair
Taskreal task
Source task
real task which of 26 mid-air interface-command gestures was drawn
Archive task
which of 26 mid-air interface-command gestures was drawn
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitSeparated
New case assumed
New people
Exact duplicates
none found D
Why
Same as GestureMidAirD1. [GestureMidAirD1: The archive description says that no person occurs in both sets (archive dataset description). The test therefore uses people absent from training. There is disagreement in the records about which participant group is training, but both describe keeping the groups separate. No shared-person concern is identified by that description.]
Source
A human-computer-interaction study of mid-air command gestures: 13 people drew each of 26 stroke-like commands (check, delete, circle, star, swipes, brackets, 3D polylines, ...) in the air, and a Leap Motion controller tracked the dominant-hand forefinger, giving a 3D trajectory per gesture.
Publication
Caputo, Prebianca, Carcangiu, Spano, Giachetti, "Comparing 3D trajectories for simple mid-air gesture recognition", Computers & Graphics 73 (2018) 17-25. Paper not read (paywalled, ScienceDirect returns HTTP 403). Layer 1 rests on the archive description page plus the authors' data/code repository https://github.com/giach68/gesturesCodes, which was inspected directly.
Full record
Benchmark task
Classify a variable-length series of one spatial coordinate of a mid-air fingertip trajectory into 26 gesture classes, training on 8 subjects and testing on 5 disjoint subjects.
Who would use it
designers of mid-air gestural interfaces (touchless control of displays, VR/AR applications)
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of 26 mid-air interface-command gestures was drawn); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selection
Leakage
passsource separation documented
Split sources
Archive page: 'Data of a same subjects does not appear in both train and test set' (8 subjects train, 5 test; the source repository lists the reverse assignment) SRS
Multichannel source
yes, one channel from a multichannel recording
The source recorded 3D position trajectory of the dominant-hand forefinger captured by a Leap Motion controller; the UCR release keeps a single channel of it. Two of the three spatial coordinates are discarded, which is severe for a stroke-shape task: several of the 26 gestures (e.g. the six poly3D variants) differ only in the ordering of movements across axes. The explicit timestamp column of the raw data is dropped, so the index is treated as uniform time. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
D1, D2 and D3 are the x, y and z coordinates of the same fingertip trajectories
Overall verdict
real task with a documented split
New case assumed
New people.
Split evidence
archive page: 'Data of a same subjects does not appear in both train and test set' (8 subjects train, 5 test; the repository lists the reverse assignment) R.
GestureMidAirD3HAR · Human motion & gesture · 208/130 train/test · length variable · 26 classesreal tasksource task labelSeparatedassumed new case: new peopletemporaltimesingle channelfamily: gesture-midair
Taskreal task
Source task
real task which of 26 mid-air interface-command gestures was drawn
Archive task
which of 26 mid-air interface-command gestures was drawn
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitSeparated
New case assumed
New people
Exact duplicates
none found D
Why
Same as GestureMidAirD1. [GestureMidAirD1: The archive description says that no person occurs in both sets (archive dataset description). The test therefore uses people absent from training. There is disagreement in the records about which participant group is training, but both describe keeping the groups separate. No shared-person concern is identified by that description.]
Source
A human-computer-interaction study of mid-air command gestures: 13 people drew each of 26 stroke-like commands (check, delete, circle, star, swipes, brackets, 3D polylines, ...) in the air, and a Leap Motion controller tracked the dominant-hand forefinger, giving a 3D trajectory per gesture.
Publication
Caputo, Prebianca, Carcangiu, Spano, Giachetti, "Comparing 3D trajectories for simple mid-air gesture recognition", Computers & Graphics 73 (2018) 17-25. Paper not read (paywalled, ScienceDirect returns HTTP 403). Layer 1 rests on the archive description page plus the authors' data/code repository https://github.com/giach68/gesturesCodes, which was inspected directly.
Full record
Benchmark task
Classify a variable-length series of one spatial coordinate of a mid-air fingertip trajectory into 26 gesture classes, training on 8 subjects and testing on 5 disjoint subjects.
Who would use it
designers of mid-air gestural interfaces (touchless control of displays, VR/AR applications)
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of 26 mid-air interface-command gestures was drawn); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selection
Leakage
passsource separation documented
Split sources
Archive page: 'Data of a same subjects does not appear in both train and test set' (8 subjects train, 5 test; the source repository lists the reverse assignment) SRS
Multichannel source
yes, one channel from a multichannel recording
The source recorded 3D position trajectory of the dominant-hand forefinger captured by a Leap Motion controller; the UCR release keeps a single channel of it. Two of the three spatial coordinates are discarded, which is severe for a stroke-shape task: several of the 26 gestures (e.g. the six poly3D variants) differ only in the ordering of movements across axes. The explicit timestamp column of the raw data is dropped, so the index is treated as uniform time. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
D1, D2 and D3 are the x, y and z coordinates of the same fingertip trajectories
Overall verdict
real task with a documented split
New case assumed
New people.
Split evidence
archive page: 'Data of a same subjects does not appear in both train and test set' (8 subjects train, 5 test; the repository lists the reverse assignment) R.
real task which of six wrist gestures was performed
Archive task
which of six wrist gestures was performed
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitConcern
New case assumed
New people
Exact duplicates
none found D
Why
The same four users performed gestures on two occasions; training uses the first occasion and testing the second (Dau et al. 2019). Every test user is also in training, so the score can rest on per-user motion signatures and does not measure recognition for new wearers of the watch; GesturePebbleZ2 is the subject-disjoint version.
Source
Mezari and Maglogiannis studied whether a commodity smartwatch can recognise natural hand gestures well enough to drive everyday device control, using the watch as the sensor and a phone as the processing unit (abstract, read in full: "Motion data produced by the accelerometer of a smart watch can be utilized in hand gesture re…
Publication
Mezari, Maglogiannis, "Gesture recognition using symbolic aggregate approximation and dynamic time warping on motion data", PervasiveHealth/PETRA (11th EAI Int. Conf. on Pervasive Computing Technologies for Healthcare), ACM 2017. Full text not read (closed access; OpenAlex reports no OA copy); abstract read in full via the OpenAlex API. The collection details above come from the UCR briefing document, which was read.
Full record
Benchmark task
Classify a variable-length z-axis wrist-accelerometer series as one of six gestures, under the split described above.
Who would use it
smartwatch-based gesture control of devices; the source group works on pervasive computing for healthcare, and the paper appeared at a health-oriented pervasive computing conference
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of six wrist gestures was performed); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selection
Leakage
faildocumented or data-verified shared source
Split sources
Dau et al. 2019 (p. 10): train = recording session 1, test = session 2 of the same four users SS
Multichannel source
yes, one channel from a multichannel recording
The source recorded three-axis acceleration from a Pebble smartwatch worn on the wrist; the UCR release keeps a single channel of it. Two of the three accelerometer axes are dropped, so gestures distinguished mainly by movement in the other planes lose their discriminating evidence; gesture duration is preserved (variable length) but the NaN padding must be handled by the user. Subject and session identity are not released as fields, only implied by the split. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
the same 304 gestures under two splits (by session in Z1, by subject in Z2)
Overall verdict
fails leakage
New case assumed
New people.
Split evidence
Dau et al. 2019 p. 10: train is session 1 and test is session 2 of the same four users, so every test user is in training (documented subject sharing) S.
GesturePebbleZ2HAR · Human motion & gesture · 146/158 train/test · length variable · 6 classesreal tasksource task labelSeparatedassumed new case: new peopletemporaltimesingle channelfamily: gesture-pebble
Taskreal task
Source task
real task which of six wrist gestures was performed
Archive task
which of six wrist gestures was performed
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitSeparated
New case assumed
New people
Exact duplicates
none found D
Why
Training uses two people and testing the other two (Dau et al. 2019). The model therefore sees unfamiliar users in the test. This differs from measuring familiar users again. The description establishes separation of people, although it does not establish separation of every other possible recording condition.
Source
Mezari and Maglogiannis studied whether a commodity smartwatch can recognise natural hand gestures well enough to drive everyday device control, using the watch as the sensor and a phone as the processing unit (abstract, read in full: "Motion data produced by the accelerometer of a smart watch can be utilized in hand gesture re…
Publication
Mezari, Maglogiannis, "Gesture recognition using symbolic aggregate approximation and dynamic time warping on motion data", PervasiveHealth/PETRA (11th EAI Int. Conf. on Pervasive Computing Technologies for Healthcare), ACM 2017. Full text not read (closed access; OpenAlex reports no OA copy); abstract read in full via the OpenAlex API. The collection details above come from the UCR briefing document, which was read.
Full record
Benchmark task
Classify a variable-length z-axis wrist-accelerometer series as one of six gestures, under the split described above.
Who would use it
smartwatch-based gesture control of devices; the source group works on pervasive computing for healthcare, and the paper appeared at a health-oriented pervasive computing conference
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of six wrist gestures was performed); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selection
Leakage
passsource separation documented
Split sources
Dau et al. 2019 (p. 10): two users train, the other two users test SS
Multichannel source
yes, one channel from a multichannel recording
The source recorded three-axis acceleration from a Pebble smartwatch worn on the wrist; the UCR release keeps a single channel of it. Two of the three accelerometer axes are dropped, so gestures distinguished mainly by movement in the other planes lose their discriminating evidence; gesture duration is preserved (variable length) but the NaN padding must be handled by the user. Subject and session identity are not released as fields, only implied by the split. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
The source data were staged by the method developers (two actors, one session) and no outside user is named; that they were made to demonstrate a method is inferred R.
SplitConcern
New case assumed
New people or recording occasions
Exact duplicates
none found D
Why
All 200 series come from one recording session of two actors, so train and test necessarily share that session and the split cannot test newly collected recordings. Which actor is on which side is not recorded. Having only two actors also limits how broadly results can be interpreted.
Source
A gesture-recognition study: two actors (one male, one female) either draw a replica gun from a hip holster and aim, or merely point with the index finger. Video tracking extracts the centroid of the right hand; the task is to tell the two gestures apart from the motion trajectory.
Classify a length-150 hand-motion series as gun-draw or point.
Who would use it
Nominal surveillance motivation ("video surveillance domain"); no user is named, and the data were presumably created as a research benchmark R, so the "original problem" is already TSC-shaped.
Task construction
failtoy source task
Benchmark target: Gun-draw vs pointing. Source target or purpose: gesture performed (gun-draw vs point), staged at UCR by the method developers. Label attached by: Collectors (= curators). Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Archive default split, 50 train / 150 test; who built it and by what rule is undocumented (the source evaluated by leave-one-out over all 200) RSR
Multichannel source
yes, one channel from a multichannel recording
The source recorded video of the actor and tracked the right-hand centroid in X and Y; the source authors themselves posed the task on the X-coordinate only, so the release keeps one channel of a two-channel trajectory. The underlying video is richer than the released series; no multivariate version of it is in UEA.
the 2003 GunPoint recording and the 2018 re-recording of the same two actors, pooled and relabelled three ways
Overall verdict
fails task construction and leakage
New case assumed
New people or recording occasions.
Split evidence
all 200 series come from one session of two actors, so train and test share that session S; per-actor split counts are unknown R. Staged at UCR in 2003 by the archive's curators, two actors S; made to demonstrate similarity methods rather than for an external user R.
toy task GunPoint re-staged for the 2018 archive release
Archive task
Gun-draw vs pointing, pooled over actors and years
Label attached by
Collectors (= curators)source taskS
Verdict
failstoy source task
The source data were staged or generated to demonstrate a method.
SplitConcern
New case assumed
New people or recording occasions
Exact duplicates
none found D
Why
Each class pools both actors and both recording years (Dau et al. 2019), and a check of the shipped files shows every actor x year combination in both train and test, so the same two people and the same two recording sessions appear on both sides of the split. The test presents further actions from those recordings; it does not establish performance on unfamiliar people or on measurements collected on a new occasion.
Source
In 2018 the creators of the 2003 GunPoint data set re-recorded it with the same two actors. UCR briefing Sec. VI-C-1, p. 9: "The original GunPoint data set was created by current authors Ratanamahatana and Keogh in 2003...
Publication
UCR briefing document (Dau et al., arXiv:1810.07758) Sec. VI-C-1, written by the data's own creators, read in full; plus the timeseriesclassification.com description page, which repeats the same account.
Full record
Benchmark task
Classify a length-150 hand-motion series as gun-draw or point, where each class pools four variants (female/male x 2003/2018).
Who would use it
nominally surveillance-style gesture discrimination; in practice a research benchmark, here specifically constructed to test invariance to actor and to recording session
Task construction
failtoy source task
Benchmark target: Gun-draw vs pointing, pooled over actors and years. Source target or purpose: GunPoint re-staged for the 2018 archive release. Label attached by: Collectors (= curators). Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question.
Data origin
temporalindex: time
Archive changes
channel-selection
Leakage
faildocumented or data-verified shared source
Split sources
Dau et al. 2019 (Sec. VI-C-1): every class pools both actors and both recording years (FG03, MG03, FG18, MG18 vs the Point variants); how the 135/316 split was drawn is not stated SR
Multichannel source
yes, one channel from a multichannel recording
The source recorded video of the two actors performing gun-draw and point actions; the released signal is the x-coordinate of the hand centroid per frame, one coordinate of a 2D hand trajectory, and the rest of the video is discarded. Merging the 2003 and 2018 sessions adds a second source of variation (equipment, tracking, 15 years of the actors) that is inside the data rather than controlled. The 2003 source tracked the hand in X and Y (GunPoint archive page) and both recordings formed each series from the X-coordinate alone, so the univariate release follows the sources' own choice.
the 2003 GunPoint recording and the 2018 re-recording of the same two actors, pooled and relabelled three ways
Overall verdict
fails task construction and leakage
New case assumed
New people or recording occasions.
Split evidence
Dau et al. Sec. VI-C-1: every class pools both actors and both recording years S; matching each series to its copies in GunPointMaleVersusFemale and GunPointOldVersusYoung gives its actor and year and shows every actor x year combination in both train and test D. How the 135/316 split was drawn is not documented.
Open questions
Concern rests on the measured presence of both actors and both recording sessions in train and test (series matched to the sibling files), not on an exact match; how the 135/316 split was drawn is not stated.
failstoy source task · instance identity as label (primary: instance identity as label)
The source data were staged or generated to demonstrate a method.
SplitConcern
New case assumed
New people or recording occasions
Exact duplicates
none found D
Why
The two classes distinguish the two actors, and recordings from both years are pooled (Dau et al. 2019). A model can recognise these particular people to predict the label. The audit reports no new recording occasion in testing. The result also does not establish the same distinction for other people.
Source
In 2018 the creators of the 2003 GunPoint data set re-recorded it with the same two actors. UCR briefing Sec. VI-C-1, p. 9: "The original GunPoint data set was created by current authors Ratanamahatana and Keogh in 2003...
Publication
UCR briefing document (Dau et al., arXiv:1810.07758) Sec. VI-C-1, written by the data's own creators, read in full; plus the timeseriesclassification.com description page, which repeats the same account.
Full record
Benchmark task
Classify a length-150 hand-motion series by which actor produced it (female class: FG03, FP03, FG18, FP18; male class: MG03, MP03, MG18, MP18), across both actions and both recording years.
Who would use it
no external stakeholder: this is a curator-constructed probe of whether a classifier can pick up actor build and posture from the hand trajectory
Task construction
failtoy source task · instance identity as label (primary: instance identity as label)
Benchmark target: Actor sex; confounded with identity (two actors). Source target or purpose: Gun-draw vs pointing action. Label attached by: Collectors (= curators). Mechanism 'instance identity as label': with one actor per class, the class is which of the two actors produced the series. Mechanism 'toy source task': the data were staged by method developers and the classes chosen to demonstrate a method, not to answer an external question. Listed under two mechanisms; counted once, under 'instance identity as label'.
Data origin
temporalindex: time
Archive changes
channel-selection
Leakage
faildocumented or data-verified shared source
Split sources
Dau et al. 2019 (Sec. VI-C-1): both recording years of each actor are pooled; how the 135/316 split was drawn is not stated SS
Multichannel source
yes, one channel from a multichannel recording
The source recorded video of the two actors performing gun-draw and point actions; the released signal is the x-coordinate of the hand centroid per frame; the UCR release keeps a single channel of it. The 2D hand trajectory is reduced to its x-coordinate and the rest of the video is discarded. Merging the 2003 and 2018 sessions adds a second source of variation (equipment, tracking, 15 years of the actors) that is inside the data rather than controlled. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
the 2003 GunPoint recording and the 2018 re-recording of the same two actors, pooled and relabelled three ways
Overall verdict
fails task construction and leakage
New case assumed
New people or recording occasions.
Split evidence
both recording sessions (2003 and 2018) of each actor are pooled on both sides, so the test presents no new session of a known actor S.
Label
Dau et al. Sec. VI-C-1: the 2003 creators (Ratanamahatana & Keogh) re-recorded the same two actors in 2018 and posed the actor task themselves S. The 2018 actor label was posed by the original collectors (Ratanamahatana & Keogh), who are also the archive's curators; with two actors the sex label is confounded with identity S.
Open questions
Label posed by the collectors (= curators) in 2018; listed under both instance identity and toy source.
GunPointOldVersusYoungHAR · Human motion & gesture · 136/315 train/test · length 150 · 2 classesfailsmetadata as label + toy source taskConcernshared sourcetemporaltimesingle channelfamily: gunpointopen question
Taskfails
Source task
toy task Gun-draw vs pointing action
Archive task
Recording session: 2003 vs 2018
Label attached by
Collectors (= curators)metadataS
Verdict
failsmetadata as label · toy source task (primary: metadata as label)
The source data were staged or generated to demonstrate a method.
SplitConcern
New case assumed
New recordings for each age condition
Exact duplicates
none found D
Why
The label distinguishes the original recording years (Dau et al. 2019). Both sets use actions recorded on those occasions. Differences in equipment or recording conditions could help the model along with age-related changes. Further recordings under each age condition would be needed to check whether the result reflects age rather than those occasions.
Source
In 2018 the creators of the 2003 GunPoint data set re-recorded it with the same two actors. UCR briefing Sec. VI-C-1, p. 9: "The original GunPoint data set was created by current authors Ratanamahatana and Keogh in 2003...
Publication
UCR briefing document (Dau et al., arXiv:1810.07758) Sec. VI-C-1, written by the data's own creators, read in full; plus the timeseriesclassification.com description page, which repeats the same account.
Full record
Benchmark task
Classify a length-150 hand-motion series as coming from the 2003 or the 2018 recording session, pooling both actors and both actions on each side.
Who would use it
no external stakeholder: a curator-constructed probe of whether the two recording sessions are distinguishable
Task construction
failmetadata as label · toy source task (primary: metadata as label)
Benchmark target: Recording session: 2003 vs 2018. Source target or purpose: Gun-draw vs pointing action. Label attached by: Collectors (= curators). Mechanism 'metadata as label': the archive derives the classes from a recording attribute. Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question. Listed under two mechanisms; counted once, under 'metadata as label'.
Data origin
temporalindex: time
Archive changes
channel-selection
Leakage
faildocumented or data-verified shared source
Split sources
Dau et al. 2019 (Sec. VI-C-1): each class is one recording year with both actors and both actions; how the 136/315 split was drawn is not stated SS
Multichannel source
yes, one channel from a multichannel recording
The source recorded video of the two actors performing gun-draw and point actions; the released signal is the x-coordinate of the hand centroid per frame; the UCR release keeps a single channel of it. The 2D hand trajectory is reduced to its x-coordinate and the rest of the video is discarded. Merging the 2003 and 2018 sessions adds a second source of variation (equipment, tracking, 15 years of the actors) that is inside the data rather than controlled. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
the 2003 GunPoint recording and the 2018 re-recording of the same two actors, pooled and relabelled three ways
Overall verdict
fails task construction and leakage
New case assumed
New recordings for each age condition.
Split evidence
the class is the recording session, so every train and test series of a class is a slice of the same session; no independent unit exists S.
Label
as GunPointMaleVersusFemale: the session task was posed by the people who recorded both sessions S. The session label (2003 vs 2018) was posed by the original collectors, who are also the archive's curators; equipment and recording-condition differences between the two sessions are available to the model S.
Open questions
Label posed by the collectors (= curators) in 2018; listed under both metadata and toy source.
HamSPECTRO · Spectroscopy & chemistry · 109/105 train/test · length 431 · 2 classesreal tasksource task labelUnclearassumed new case: new physical samplesordered, non-temporalspatial position
Taskreal task
Source task
real task country of origin of a dry-cured ham (Spanish vs French)
Archive task
country of origin of a dry-cured ham (Spanish vs French)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New physical samples
Exact duplicates
none found D
Why
Each physical ham supplies several image measurements. We do not know whether all measurements of a ham were kept in one set. The test could therefore contain another measurement of a familiar ham. Not finding very similar pairs in our similarity check does not prove that different hams were used.
Source
A provenance-verification study for protected regional foods, run at the Institute of Food Research, Norwich: can Spanish and French dry-cured hams be distinguished by their protein profiles? This is not spectroscopy - the measurement is SDS-PAGE gel electrophoresis.
Publication
Olias, Maldonado, Radreau, Le Gall, Mulholland, Colquhoun, Kemsley, "Sodium dodecyl sulphate-polyacrylamide gel electrophoresis of proteins in dry-cured hams: data registration and multivariate analysis across multiple gels", Electrophoresis 27(7), 1288-1299, 2006, doi:10.1002/elps.200500466. Abstract read via the Europe PMC REST API; full text not reached. The donor's own page, IFR Analytical Sciences "Example Datasets for Download", describes the release: "214 Lanes of SDS-PAGE data, from 19 Spanish and 18 French dry-cured hams; data matrix size [214 x 431]. Triplicate lanes of duplicate extractions (giving ~6 data vectors per sample), pre-processed and aligned". [214 x 431] matches the archive exactly (109 + 105 = 214 series of length 431).
Full record
Benchmark task
Classify a univariate length-431 registered SDS-PAGE lane profile as Spanish or French ham.
The benchmark label is the target the source study posed (country of origin of a dry-cured ham (Spanish vs French)); label origin: source task.
Data origin
ordered, non-temporalindex: spatial position
Index is position along a column profile or gel lane, not time.
Archive changes
spectrum-as-seriesspatial-resampling
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RDR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New physical samples.
Split evidence
37 hams yield 214 lanes (about six per ham); the split is not documented as ham-disjoint; a nearest-neighbour check found no near-duplicate lane pairs across the split R+D.
HandOutlinesIMAGE · Medical imaging · 1,000/370 train/test · length 2709 · 2 classesreal tasksource task labelUnclearassumed new case: new patientsordered, non-temporalcontour arc-lengthfamily: phalanx-radiographsopen question
Taskreal task
Source task
real task whether the automatically extracted hand outline was judged correct (verification stage of the bone-ageing pipeline)
Archive task
Whether automatic hand outline is judged correct
Label attached by
Source studysource taskS
Verdict
real task
The source pipeline posed hand-outline verification as one of its stages; the archive keeps that target as the label.
SplitUnclear
New case assumed
New patients
Exact duplicates
none found D
Why
The outlines come from hand X-rays. We do not know whether one patient contributed several images or whether those images were kept together. The test could contain a new image of a familiar patient. Patient identifiers or an explicit patient grouping rule are needed to check this.
Source
The first stage of the same UEA bone-ageing pipeline: before any bone can be staged, the hand itself must be segmented from the radiograph, and bad segmentations must be caught.
Publication
Davis et al., "On the Segmentation and Classification of Hand Radiographs", International Journal of Neural Systems 22(5), 2012 - not read (World Scientific paywall; the UEA project page linked from the archive no longer carries the project content). Pipeline context taken from Bagnall & Davis, arXiv:1406.4781 (read), which covers the bone stage rather than the hand-outline stage.
Full record
Benchmark task
Classify a length-2709 hand-outline series as a correct or incorrect segmentation.
Who would use it
developers and users of an automated bone-ageing system, who need to reject failed hand segmentations before downstream bone measurement
Task construction
passnone (source task label)
Benchmark target: Whether automatic hand outline is judged correct. Source target or purpose: whether the automatic hand outline was judged correct (verification stage of the bone-ageing pipeline). Label attached by: Source study. The archive keeps the pipeline's own verification label, so task construction passes.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
the same ~1300 hand radiographs of the same subjects; different bones and labels (age band, Tanner-Whitehouse stage, segmentation correctness); PhalangesOutlinesCorrect pools three siblings
Overall verdict
real task; leakage unknown
New case assumed
New patients.
Split evidence
one outline per radiograph; subject overlap across the split undocumented R.
Label
the correct/incorrect label was produced inside Davis & Bagnall's own bone-ageing pipeline as its hand-outline verification stage; it describes the segmentation, not the patient S.
Open questions
Patient-level separation, the evaluators' adjudication rule and the outline-to-series mapping (IJNS 2012, not read) remain undocumented.
HapticsMOTION · Human motion & gesture · 155/308 train/test · length 1092 · 5 classesfailsinstance identity as labelUnclearassumed new case: new recording occasionstemporaltimesingle channelopen question
Taskfails
Source task
none none documented; graphical-password (passgraph) entry on a touchscreen, presumably for authentication
Archive task
which of five users entered the passgraph (closed-set identification)
Label attached by
Unknowninstance identitySR
Verdict
failsinstance identity as label
No source study is documented (no donor, link or paper). The archive page describes graphical-password entry, so an authentication purpose is a reading of that one sentence R; whether the collectors posed any classification target is unknown.
SplitUnclear
New case assumed
New recording occasions
Exact duplicates
none found D
Why
The labels identify users from repeated measurements. We do not know whether the test measurements were collected on occasions separate from training. Familiar identities are expected, but the model might also recognise conditions specific to one recording. Identification when a user is measured again remains unverified.
Source
Behavioural biometrics on a touchscreen: five people entering their graphical password (passgraph), X-axis movement only; the archive page is the only description and no donor, link or paper exists.
Publication
timeseriesclassification.com description page only (read 2026-08). No primary source found; the archive page lists no donor and no data-source link.
Full record
Benchmark task
Classify a length-1092 single-axis touchscreen trajectory as one of five users. 155 train / 308 test.
Who would use it
Designers of graphical/behavioural authentication systems, who would use such a model as a second factor (is this the enrolled user drawing the pattern?).
Task construction
failinstance identity as label
Benchmark target: which of five people entered the passgraph. No source study is documented. Label attached by: Unknown. Mechanism 'instance identity as label': the class is the person who produced the series.
Data origin
temporalindex: time
Archive changes
channel-selectionz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded finger trajectory on a touchscreen while entering a graphical password; the UCR release keeps a single channel of it. A passgraph entry is a 2D gesture; only the X coordinate is released, so the drawn shape is only half observed and two different passgraphs that differ mainly in Y become indistinguishable. Pressure and timing metadata, which is what makes touch biometrics work in the literature, are not present either. Z-normalisation removes absolute screen position and stroke scale. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
Dataset family
none from the same source task
Overall verdict
fails task construction (leakage unknown)
New case assumed
New recording occasions.
Split evidence
five users with repeated entries; no held-out-session evaluation is documented, so whether the test presents a new session of a known user is unknown R.
Label
Archive text: five people entering their passgraph; the label (user) is the biometric target the collection implies; no source documents this, so it is our reading, not a sourced fact I. No primary source; everything rests on one archive sentence. Whether the five users entered the same passgraph is undocumented; if not, the classifier may recognise the drawn shape rather than the person R.
Open questions
Authentication recast as five-way identification (archive page only); counted as instance identity as label. Whether the five users drew the same passgraph is undocumented.
real task herring stock of origin (North Sea vs Thames)
Archive task
herring stock of origin (North Sea vs Thames)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New physical specimens
Exact duplicates
none found D
Why
The cases are outlines derived from fish ear-bone images. The description does not establish how physical specimens and any repeated views were assigned to the two sets. New-specimen evaluation therefore cannot be verified; repeated views are a possibility to check, not an observed overlap.
Source
Herring stock identification from otolith shape - a fisheries-management problem. The archive page sets out the domain background: "Otoliths contain information that can be used by 'expert readers' to determine several key factors important in managing fish stock.
Publication
Mapp, Fisher, Bagnall, Lines, Warne & Scutt Phillips, "Clupea Harengus: Intraspecies Distinction using Curvature Scale Space and Shapelets - Classification of North-sea and Thames Herring using Boundary Contour of Sagittal Otoliths", ICPRAM 2013, pp. 138-143 - title, authorship and abstract read via the SciTePress record; the full paper was not reached, and the UEA otolith page linked from the archive would not serve (TLS certificate mismatch).
Full record
Benchmark task
Classify a length-512 otolith-boundary series as North Sea or Thames herring.
Who would use it
fisheries scientists and managers estimating stock composition of catches
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (herring stock of origin (North Sea vs Thames)); label origin: source task.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New physical specimens.
Split evidence
outline extraction unverified, so how fish and otoliths map to series and to the split is unknown R.
Whole-house aggregate vs summed dryer/washer channels
Label attached by
CuratorsredefinedS
Verdict
failstask replacement
The source collected the data but posed no classification target.
SplitConcern
New case assumed
New households
Exact duplicates
none found D
Why
All cases are pieces of one household's recording (REFIT House 20), so training and testing share the same home, its appliances and its routine. The test measures recognition within that home; performance in another home, which load monitoring needs, is not tested.
Source
The data come from the REFIT electrical load measurements study, which continuously monitored whole-house aggregate load plus nine individual appliances in each of 20 UK households at 8-second intervals over two years.
Publication
REFIT project ("Personalised Retrofit Decision Support Tools for UK Homes using Smart Home Technology"); primary data descriptor read: Murray, Stankovic & Stankovic, "An electrical load measurements dataset of United Kingdom households from a two-year longitudinal study", Scientific Data 4, 160122 (2017), Abstract: "The REFIT electrical load measurements dataset described in this paper includes whole house aggregate loads and nine individual appliance measurements at 8-second intervals per house, collected continuously over a period of two years from 20 houses." Curation (House 20; aggregate vs tumble-dryer+washing-machine) is documented only on the timeseriesclassification.com page, attributed to Gharghabi, Dau and Keogh; the UCR briefing document does not discuss HouseTwenty.
Full record
Benchmark task
Classify a 2000-point power window from REFIT House 20 as whole-house aggregate load or as the combined tumble-dryer/washing-machine load.
Who would use it
REFIT targets energy feedback and retrofit advice; separating appliance load from whole-house load is the core of non-intrusive load monitoring, though the two-class version here is a curator-built benchmark rather than a deployed formulation.
Task construction
failtask replacement
Benchmark target: Whole-house aggregate vs summed dryer/washer channels. Source target or purpose: REFIT load monitoring; no class target. Label attached by: Curators. Mechanism 'task replacement': the archive replaces the source's label with a different task, or defines classes by its own rule where the source posed none.
Data origin
temporalindex: time
Archive changes
windowingconcatenation
Leakage
faildocumented or data-verified shared source
Split sources
Archive default split; who built it and by what rule is not documented RS
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails task construction and leakage
New case assumed
New households.
Split evidence
every series comes from one household, REFIT House 20 (archive page: 'We use data of House 20 to make one dataset') S; train and test are windows of that household's continuous recording, so no other home is tested.
Label
aggregate-vs-appliance classes defined on the archive page, edited by the curators (Gharghabi, Dau, Keogh) from REFIT House 20 channels S. Class 2 is a curator-constructed sum of the tumble-dryer and washing-machine channels of REFIT House 20 against the whole-house aggregate S.
Open questions
Whether test windows come from a later period than training windows is not documented; it would not change the verdict, because no other household is tested.
InlineSkateMOTION · Human motion & gesture · 100/550 train/test · length 1882 · 7 classeslabel meaning undocumentedcounted as a real taskConcernduplicates 1.1%temporaltimesingle channelopen question
Tasklabel meaning undocumented
Source task
undocumented no classification target documented (sports-medicine study of inline-skating technique)
Archive task
one of seven classes, assumed by the curators to be seven individuals (unconfirmed)
Label attached by
CuratorsundocumentedR
Verdict
label meaning undocumented
No source documents a classification target; the class variable was introduced at donation or by the curators with undocumented meaning.
SplitConcern
New case assumed
New recording occasions
Exact duplicates
6 of 550 test series (1.1%) exactly match a training series D
Why
Six of the 550 test series are value-identical to training series. If the classes are individuals, training and test cycles may also come from the same treadmill run and share its person, equipment and conditions, but no source confirms this. The test does not show whether recognition works on another recording, and what the class labels identify is itself unconfirmed.
Source
Sports-medicine biomechanics: EMG and kinematic recordings of professional inline speed skaters on a treadmill (Hoos, Marburg), used by Morchen for pattern discovery in his PhD thesis; the archive does not know what its seven classes are.
Publication
timeseriesclassification.com description page (read 2026-08), which credits Olaf Hoos (Univ. Marburg, Sports Medicine) and Fabian Morchen's PhD thesis "Time Series Knowledge Mining" (Univ. Marburg). Neither the thesis nor the paper behind the archive's `data_source_link` (idb.csie.ncku.edu.tw/.../p668.pdf) could be retrieved - the link failed to fetch - so no primary text was read.
Full record
Benchmark task
Classify a length-1882 ankle-angle series from one inline-skating movement cycle into one of seven classes (assumed by the curators, unconfirmed, to be seven individuals). 100 train / 550 test.
Who would use it
sports scientists and coaches analysing skating technique and muscle coordination
Task construction
passnone (label meaning undocumented)
Benchmark target: one of seven classes, assumed by the curators to be seven individuals (unconfirmed). The source posed no classification target and no source says what the archive's classes mean, so original task and label origin are recorded as undocumented. We do not count it among the constructed tasks, because no source shows that the archive constructed the label; it is counted with the real tasks and flagged as an open question.
failexact cross-split matches (6 of 550 test series)
Split sources
Archive default split; who built it and by what rule is not documented RD
Multichannel source
yes, one channel from a multichannel recording
The source recorded surface EMG of seven leg muscles plus kinematic measurement of the right body side during treadmill inline speed skating; the UCR release keeps a single channel of it. The original recording is multivariate (7 EMG channels plus kinematics); only a single kinematic channel, the ankle angle, survives - so the muscle-activation content that motivated the original study is entirely absent. Truncation to a common length cuts the end off longer movement cycles, which biases against slower cycles, and z-normalisation removes the joint-angle range of motion. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
Dataset family
none from the same source task
Overall verdict
label meaning undocumented (counted with the real tasks); fails leakage
New case assumed
New recording occasions.
Split evidence
6 of 550 test series are value-identical to training series D; if the classes are individuals, every cycle of a person may come from the same recording session, so train and test windows would be cycles of the same treadmill run R.
Label
the class variable was introduced at donation and the curators state they do not know what it is; whether the donors (Morchen, Hoos) or the archive defined it is undocumented R. The archive assumes the seven classes are seven individuals 'although this is not confirmed'; the label meaning is undocumented, so the dataset is not counted among the constructed tasks R.
Open questions
The archive is not sure what the seven classes are (assumed individuals). Not counted among the constructed tasks.
InsectEPGRegularTrainEPG · Biology & ecology · 62/249 train/test · length 601 · 3 classesreal tasksource task labelUnclearassumed new case: new insects or recordingstemporaltimefamily: insect-epgopen question
Taskreal task
Source task
real task insect feeding-behaviour state (six expert-defined EPG waveform states)
Archive task
one of three feeding-behaviour classes (the archive keeps three of the six states; the mapping is undocumented)
Label attached by
Source studysource taskSR
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New insects or recordings
Exact duplicates
none found D
Why
The cases are windows cut from long EPG recordings of individual insects. No insect or recording identifier ships with the data, so we cannot check whether the windows of one insect were kept in one set; windows of the same insect on the same plant, minutes apart, could sit on both sides, but this is not established. The test set is shared with InsectEPGSmallTrain, whose smaller training set contains exact matches of 15 test series; this training set contains none.
Source
Electrical penetration graph recordings of the Asian citrus psyllid probing a plant (Willett et al. 2016), made to automate the classification of feeding behaviour that is the bottleneck in screening citrus for resistance to citrus greening.
Publication
Willett DS, George J, Willett NS, Stelinski LL, Lapointe SL, "Machine Learning for Characterization of Insect Vector Feeding", PLoS Computational Biology 12(11): e1005158, 2016. Read 2026-08 via the PLOS article page. Labels: "Waveforms were classified by visual inspection by a trained expert according to previous reports", assigned second-by-second across roughly 470 hours of recordings from nine citrus varieties. Underlying data: Dryad, doi:10.5061/dryad.4931c, "Labelled EPG data from recordings of Asian Citrus Psyllid (Diaphorina citri) feeding on citrus cultivars".
Full record
Benchmark task
Classify a length-601 EPG voltage window into one of 3 insect-feeding-behaviour classes. 62 train / 249 test.
Who would use it
Citrus growers and plant-breeding programmes screening cultivars for resistance to psyllid feeding, and entomologists studying pathogen transmission.
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (insect feeding-behaviour state (six expert-defined EPG waveform states)); label origin: source task. The archive coarsened the target (six expert waveform states collapsed to three by the curators); a coarsened source target still counts as source task.
Data origin
temporalindex: time
Archive changes
windowingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Dau et al. 2019 (p. 11): one 249-series test set shared with InsectEPGSmallTrain; 62 training series SR
windows are cut from long continuous recordings of individual insects; whether windows of one insect sit on both sides is undocumented and cannot be checked from the released files R.
Label
six expert waveform states collapsed to three by the curators R. Six expert-defined waveform states collapsed to three by the curators (coarsened source target) R.
Open questions
Six expert states collapsed to three by the curators (coarsened); not counted among the constructed tasks. Shared-insect windows across the split are plausible from the recording design but not documented, so Unclear rather than Concern.
real task insect feeding-behaviour state (six expert-defined EPG waveform states)
Archive task
one of three feeding-behaviour classes (the archive keeps three of the six states; the mapping is undocumented)
Label attached by
Source studysource taskSR
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitConcern
New case assumed
New insects or recordings
Exact duplicates
15 of 249 test series (6%) exactly match a training series D
Why
The cases are windows cut from long EPG recordings of individual insects. No insect or recording identifier ships with the data, so whether windows of one insect sit on both sides is not established. The value check finds 15 of the 249 test series identical to series in this small training set, so a model can score them by recall.
Source
Electrical penetration graph recordings of the Asian citrus psyllid probing a plant (Willett et al. 2016), made to automate the classification of feeding behaviour that is the bottleneck in screening citrus for resistance to citrus greening; small-training-set variant.
Publication
Willett DS, George J, Willett NS, Stelinski LL, Lapointe SL, "Machine Learning for Characterization of Insect Vector Feeding", PLoS Computational Biology 12(11): e1005158, 2016. Read 2026-08 via the PLOS article page. Labels: "Waveforms were classified by visual inspection by a trained expert according to previous reports", assigned second-by-second across roughly 470 hours of recordings from nine citrus varieties. Underlying data: Dryad, doi:10.5061/dryad.4931c, "Labelled EPG data from recordings of Asian Citrus Psyllid (Diaphorina citri) feeding on citrus cultivars".
Full record
Benchmark task
Classify a length-601 EPG voltage window into one of 3 insect-feeding-behaviour classes. 17 train / 249 test.
Who would use it
Citrus growers and plant-breeding programmes screening cultivars for resistance to psyllid feeding, and entomologists studying pathogen transmission.
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (insect feeding-behaviour state (six expert-defined EPG waveform states)); label origin: source task. The archive coarsened the target (As InsectEPGRegularTrain); a coarsened source target still counts as source task.
Data origin
temporalindex: time
Archive changes
windowingz-normalization
Leakage
failexact cross-split matches (15 of 249 test series)
Split sources
Dau et al. 2019 (p. 11): the same 249-series test set as InsectEPGRegularTrain; 17 training series SDR
windows are cut from a small number of long continuous recordings of individual insects; windows of the same insect on the same plant, minutes apart, can sit on both sides, but no identifier ships to check it R.
Label
the source study's expert annotations ('classified by visual inspection by a trained expert') S. Six expert-defined waveform states collapsed to three by the curators (coarsened source target; mapping undocumented) R.
Open questions
Six expert states collapsed to three by the curators (coarsened); not counted among the constructed tasks.
InsectWingbeatSoundSENSOR · Biology & ecology · 220/1,980 train/test · length 256 · 11 classesreal tasksource task labelUnclearassumed new case: new insect groups or recordingsordered, non-temporalfrequency (power spectrum)family: insect-wingbeatopen question
Taskreal task
Source task
real task insect species and sex from the wingbeat sound (11 classes)
Archive task
insect species and sex from the wingbeat sound (11 classes)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New insect groups or recordings
Exact duplicates
none found D
Why
The source recorded insects in groups inside cages. Cases from one group can share its insects and recording conditions. The files do not identify those groups, so we cannot check whether a whole group was kept in one set. Shared groups are possible, but have not been verified.
Source
UCR's computational entomology group built cheap pseudo-acoustic optical sensors — a laser line pointed at a phototransistor array — that record the light fluctuations caused by an insect's wings occluding the beam as it flies through.
Publication
Chen, Why, Batista, Mafra-Neto & Keogh, "Flying Insect Classification with Inexpensive Sensors", J. Insect Behavior 27(5):657-677, 2014 (arXiv:1403.2654, read in full). On the sensor: "design of the sensor consists of a phototransistor array which is connected to an electronic board, and a laser line pointing at the phototransistor array. When an insect flies across the laser beam, its wings partially occlude the light, causing small light fluctuations." On why the frequency domain: "The vast majority of attempts to classify insects by their flight sounds have explicitly or implicitly used just the wingbeat frequency... The Bayes error rate to classify the three species in Figure 1.I using just the wingbeat frequency is 19.13%; however, as we shall see below... by using the additional features from the signal, we can obtain [better results]." The detector stage is "a nearest neighbor classifier based on the frequency spectrum". That the ARCHIVE series are the spectrum rather than the raw sound is stated by the secondary dataset documentation (fda.tsc R package, InsectWingbeatSoundTSC: "The data in the archive is the power spectrum of the sound"; 2200 curves of 256 points, first 220 train) and by the timeseriesclassification.com description of the sibling InsectWingbeat dataset; the UCR description.php page for InsectWingbeatSound itself renders empty (checked 2026-08-18).
Full record
Benchmark task
Classify a univariate length-256 power spectrum of an insect flight sound into one of 11 species/sex classes.
Who would use it
public-health vector control (mosquito surveillance) and agricultural pest monitoring
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (insect species and sex from the wingbeat sound (11 classes)); label origin: source task.
Data origin
ordered, non-temporalindex: frequency (power spectrum)
Index is frequency: each series is the power spectrum of a wingbeat sound recording, so the ordered axis is spectral, not time.
Archive changes
spectrum-as-seriesz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
no
Dataset family
insect-wingbeat with InsectWingbeatUEA
the same optical insect-sensor programme (low confidence: the recordings are not documented as shared)
Overall verdict
real task; leakage unknown
New case assumed
New insect groups or recordings.
Split evidence
instances of a class come from a few caged cohorts of 20-40 insects; cohort identity is not recoverable R. The series is a 256-bin power spectrum of the insect flight sound (index = frequency); filed as ordered, non-temporal R.
Open questions
Index is frequency; a reader who regards a power spectrum as a time axis would count it as temporal.
ItalyPowerDemandSENSOR · Energy & appliances · 67/1,029 train/test · length 24 · 2 classesfailsmetadata as labelRelated periodsassumed new case: new days or later periodstemporaltimeopen question
Taskfails
Source task
none None; utility demand records
Archive task
Cold season (Oct-Mar) vs warm season (Apr-Sep)
Label attached by
CuratorsmetadataR
Verdict
failsmetadata as label
The source collected the data but posed no classification target.
SplitRelated periods
New case assumed
New days or later periods
Exact duplicates
none found D
Why
Training and test days can share the same season and similar weather. Different days are tested, so this is not automatically leakage. However, the split does not specifically test a later period or another grid.
Source
Hourly electrical power demand of Italy in 1997, one 24-point curve per day; the utility recorded demand, and Keogh et al. (ICDM 2006) turned the days into a cold-season vs warm-season problem from the timestamps.
Publication
Keogh, Wei et al., "Intelligent Icons: Integrating Lite-Weight Data Mining and Visualization into GUI Operating Systems", ICDM 2006
Full record
Benchmark task
Classify a 24-point daily load curve as winter-half or summer-half of the year.
Who would use it
Utilities analyze demand curves for forecasting/planning, but season-from-load-shape is a curator-constructed task, not an operational one.
Task construction
failmetadata as label
Benchmark target: Cold season (Oct-Mar) vs warm season (Apr-Sep). Source target or purpose: None; utility demand records. Label attached by: Curators. Mechanism 'metadata as label': the archive derives the classes from a recording attribute.
Data origin
temporalindex: time
Archive changes
windowingz-normalization
Leakage
pass (related periods)different periods of one ongoing process
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails task construction
New case assumed
New days or later periods.
Split evidence
days of one country's demand in one year split at random; deployment sees new days of the same grid, but train and test days are neighbours in one year R.
Label
season label constructed by Keogh et al. (ICDM 2006) from timestamps of a utility's demand data R. The season label was constructed by Keogh et al. (ICDM 2006) from the timestamps of a utility's demand data R.
Open questions
Related periods (different days of one grid and year); whether this counts as clean is undecided.
LargeKitchenAppliancesDEVICE · Energy & appliances · 375/375 train/test · length 720 · 3 classesreal tasksource task labelUnclearassumed new case: new appliances or householdstemporaltimefamily: household-electricity-surveyopen question
Taskreal task
Source task
real task Household electricity monitoring for energy policy (Household Electricity Survey); each metered channel is annotated with its appliance, but the survey posed no classification task
Archive task
which large kitchen appliance produced the day's consumption trace (washing machine, tumble dryer, dishwasher)
Label attached by
Collectorssource taskSR
Verdict
real task
The survey's installing electricians annotated each monitored product with its appliance ("selected from a dropdown list"), so under policy L3 we count appliance-type recognition as the source's own task although the survey trained no classifier; the classification problem was posed on those annotations by the curators (Lines & Bagnall, SDM 2014; Lines 2015 thesis). Each input is an appliance trace metered on its own, so the task is appliance recognition, not whole-house disaggregation.
SplitUnclear
New case assumed
New appliances or households
Exact duplicates
none found D
Why
Same as Computers. [Computers: The source measured each physical appliance over many days. We do not know whether all days from an appliance or household were kept in one set. If the same appliance appears in both sets, the model may recognise its usual pattern. That would be a narrower test than recognising appliance types in new homes.]
Source
These series come from the UK government-sponsored household electricity monitoring study known as "Powering the Nation" (the DECC/Defra/Energy Saving Trust Household Electricity Survey).
Publication
UK Household Electricity Survey (DECC, Defra and the Energy Saving Trust, 2010-2011); Intertek Report R66141, "Household Electricity Survey: A study of domestic electrical product usage", final report issue 4. Household counts differ: the archive page, the thesis and Intertek (p. 24) give 251 households, Report 5 (p. 4) and gov.uk give 250; the 26 year-long households were sampled every 10 minutes except from late November 2010 to February 2011, when they recorded every 2 minutes (Intertek p. 64). Curated into TSC datasets by Lines & Bagnall (UEA): J. Lines, PhD thesis, UEA, 2015, Sec. 3.2.2 and Table 3.3 ("We provide five new classification problems that focus on domestic appliance classification from electricity consumption"); introduced in Lines & Bagnall, "Ensembles of elastic distance measures for time series classification", SIAM SDM 2014 (not read: no full text found).
Full record
Benchmark task
Classify a 720-point daily consumption profile as washing machine, tumble dryer or dishwasher.
Who would use it
UK energy policy (the survey); the curators' stated aim was to extract more from smart-meter data by "classifying appliances according to their electricity usage profiles". The derived task (naming a device from its own metered daily profile) is appliance recognition, not disaggregation of a whole-house load.
Task construction
passnone (source task label)
The benchmark label is the survey's appliance annotation of the metered channel (which large kitchen appliance produced the day's consumption trace (washing machine, tumble dryer, dishwasher)); label origin: source task, by the convention that a collectors' annotation of the quantity a deployed model would predict counts as the source task.
Data origin
temporalindex: time
Archive changes
windowingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split, made by the curators (Lines & Bagnall), who say these datasets were "split in a certain manner to avoid introducing bias" from one device appearing in both train and test S; the rule used is not stated and the files carry no household or device identifiers RIR
five appliance-group tasks cut from the UK Household Electricity Survey (ElectricDevices comes from a different study, the Visual Energy Trial); whether they share households or appliances is not documented
Overall verdict
real task; leakage unknown
New case assumed
New appliances or households.
Split evidence
each series is one day cut from a month (or, for some households, a year) of metering, so one appliance can supply many series R; which households, appliances and days entered the dataset is not documented. The curators say the split was made to avoid device bias, but give no rule, and the files carry no device identifiers, so device- or household-disjointness is neither documented nor refuted and sharing is not counted as leakage R. An appliance-recognition model meeting unseen appliances is the deployment case I.
Open questions
Appliance-type recognition is counted as the source task by convention (the survey annotated the channels but posed no classification task); a stricter reading files it as metadata. Appliance/household sharing inferred from the survey design but not documented, so Unclear rather than Concern.
Lightning2SENSOR · Earth & space science · 60/61 train/test · length 637 · 2 classesreal tasksource task labelUnclearassumed new case: new recorded eventstemporaltimefamily: lightningopen question
Taskreal task
Source task
real task lightning transient type (seven expert classes in the source pipeline); Sec. 5 also poses cloud-to-ground vs intra-cloud
Archive task
cloud-to-ground vs intra-cloud lightning event (inferred; the archive does not name the two classes)
Label attached by
Source studysource taskSR
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New recorded events
Exact duplicates
none found D
Why
Each series is one recorded lightning event, but no source documents how the 121 records were divided or whether events of one flash or storm sit on both sides. The source paper describes a two-class cloud-to-ground vs intra-cloud subproblem of its 143 records, evaluated by ten-fold cross-validation; dropping its 22 off-record events leaves 121, Lightning2's size, so Lightning2 is likely that set, but its fixed split is not documented. The split cannot be checked.
Source
Los Alamos researchers working with the FORTE (Fast On-orbit Recording of Transient Events) satellite needed to sort the enormous stream of recorded electromagnetic transients into physical event types (cloud-to-ground return strokes, intra-cloud events, and so on) automatically.
Publication
The archive description is the donor's own account of the preprocessing: "Data is collected with a sample rate of 50 MHz for 800 microseconds. A Fourier transform is performed on the input data to produce a spectrogram. The spectrograms are then collapsed in frequency to produce a power density time series, with 3181 samples in each time series. These are then smoothed to produce series of length 637." The underlying study is Eads, Hill, Davis, Perkins, Ma, Porter & Theiler, "Genetic algorithms and support vector machines for time series classification", Proc. SPIE 4787, 2002. That paper was read (November 2026): it describes 143 records in seven classes and, in Sec. 5, a two-class subproblem, cloud-to-ground (CG, IR, SR) vs intra-cloud (I, I2, KM) events, evaluated by ten-fold cross-validation; it gives no fixed train/test split.
Full record
Benchmark task
Classify a length-637 power-density trace of a FORTE transient into one of two classes, most likely cloud-to-ground vs intra-cloud events.
Who would use it
atmospheric-science and nuclear-test-monitoring programmes that must distinguish natural lightning transients from other RF events
Task construction
passnone (source task label)
The source study classified FORTE transients into seven expert lightning types and posed a two-class subproblem, cloud-to-ground (CG, IR, SR) vs intra-cloud (I, I2, KM) events (Eads et al. 2002, Sec. 5). Without the 22 off-record events (implied by the Sec. 4 confusion matrices) its 143 records leave 121, Lightning2's size, so the label is taken as that source task R; the archive itself does not name the classes ('Not exactly sure what the two classes are').
Data origin
temporalindex: time
Archive changes
spectrum-as-seriesresampling
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
the same FORTE campaign and pipeline; Lightning2's two classes are taken as the source's cloud-to-ground vs intra-cloud grouping of Lightning7's seven classes without off-record events (inferred)
Overall verdict
real task; leakage unknown
New case assumed
New recorded events.
Split evidence
each series is one recorded FORTE transient; no source documents how the split was built or whether events of one flash sit on both sides R. The archive's own description says 'Not exactly sure what the two classes are'; the label is taken as the source's two-class subproblem, cloud-to-ground vs intra-cloud events (Eads et al. 2002, Sec. 5) S, matched by record count R.
Open questions
The archive does not name the two classes; reading them as the source's cloud-to-ground vs intra-cloud subproblem rests on the matching record count (121) R, and the class sizes (48/73) differ by one from those the source's seven classes imply (49/72).
Lightning7SENSOR · Earth & space science · 70/73 train/test · length 319 · 7 classesreal tasksource task labelUnclearassumed new case: new recorded eventstemporaltimefamily: lightning
Taskreal task
Source task
real task lightning transient type (seven expert-defined discharge categories)
Archive task
lightning transient type (seven expert-defined discharge categories)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New recorded events
Exact duplicates
none found D
Why
Each series is one recorded lightning event, but nothing documents how the 143 records were divided. The source notes that subsequent return strokes come after initial return strokes, so strokes of one flash, or events of one storm, could sit on both sides. The split cannot be checked.
Source
FORTE satellite recordings of VHF electromagnetic transients from lightning, each labelled by domain experts with its physical discharge type and used at Los Alamos to drive a genetic-programming feature extractor.
Publication
The archive description is the donor's own account of both the preprocessing and the class physics: "Data is collected with a sample rate of 50 MHz for 800 microseconds. A Fourier transform is performed on the input data to produce a spectrogram. The spectrograms are then collapsed in frequency to produce a power density time series, with 3181 samples in each time series. These are then smoothed to produce series of length 637." On the classes, e.g.: "IR Negative Initial Return Stroke... The power waveform slowly ramps up to a level known as an attachment point, where a large surge current causes the VHF power to 'spike'. This attachment is followed by an exponentially shaped decline in the waveform." and "O Off-record: 800 microseconds was not enough to fully capture the lightning event." The underlying study is Eads, Hill, Davis, Perkins, Ma, Porter & Theiler, Proc. SPIE 4787, 2002; it was read (November 2026): Sec. 2.3 gives the seven classes and 143 records, Sec. 4.1 a ten-fold random cross-validation; it gives no fixed train/test split.
Full record
Benchmark task
Classify a length-319 power-density trace of a FORTE transient into one of seven expert-defined lightning event types.
Who would use it
atmospheric electricity research and treaty/nuclear-test monitoring programmes that must classify space-detected RF transients automatically
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (lightning transient type (seven expert-defined discharge categories)); label origin: source task.
Data origin
temporalindex: time
Archive changes
spectrum-as-seriesresampling
Leakage
unknownsource separation undocumented
Split sources
Archive default 70/73 split of the source's 143 records; it is not the evaluation protocol reported by the source, which used ten-fold random cross-validation (Eads et al. 2002, Sec. 2.3 and 4.1) S; who built the fixed split and by what rule is not documented RSR
the same FORTE campaign and pipeline; Lightning2's two classes are taken as the source's cloud-to-ground vs intra-cloud grouping of Lightning7's seven classes without off-record events (inferred)
Overall verdict
real task; leakage unknown
New case assumed
New recorded events.
Split evidence
each series is one recorded FORTE transient S; the source evaluated by random ten-fold cross-validation, and no source documents how the archive's 70/73 split was built or whether strokes of one flash sit on both sides R.
MallatSIMULATED · Simulated · 55/2,345 train/test · length 1024 · 8 classesfailstoy source taskGeneratedassumed new case: new generated examplessyntheticgenerator step
Taskfails
Source task
toy task None; wavelet test signal
Archive task
Eight generated signal variants; classes undocumented
Label attached by
Generatorsource taskR
Verdict
failstoy source task
The source data were staged or generated to demonstrate a method.
SplitGenerated
New case assumed
New generated examples
Exact duplicates
none found D
Why
This is a generated task rather than measurements from people, devices or specimens. The corresponding source-separation question is therefore not assessed here. Its split would instead need to be examined through the generation procedure; synthetic origin alone is not evidence of leakage.
Source
No real-world problem exists. The archive states only that Mallat is a simulated dataset described in Mallat, "A Wavelet Tour of Signal Processing" (1998), donated by M.
Publication
Archive page only - "Mallat is a simulated dataset described in Mallat, S. G. (1998), A Wavelet Tour of Signal Processing", donated_by "M Jeong, S Mallat", with no data_source_link and no second_link. The book's relevant section, and any paper by M. Jeong defining the eight classes, could not be located; searches returned only papers that use the dataset via the archive. Stating what the classes mean would be invention, so it is left as unknown here. What can be checked directly from the shipped data - 2400 series of length 1024, exactly 300 per class, and all eight class-mean waveforms share one piecewise-smooth base signal (deep transients around index ~300, a large excursion around index ~850-890) and differ only in the height and shape of a few local segments (chiefly around indices 160-300 and 850-900) - which is consistent with eight variants of a single test signal, but does not identify them.
Full record
Benchmark task
Classify a univariate length-1024 series into eight generated signal classes.
Who would use it
none (synthetic demo)
Task construction
failtoy source task
Benchmark target: Eight generated signal variants; classes undocumented. Source target or purpose: None; wavelet test signal. Label attached by: Generator. Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question.
Data origin
syntheticindex: generator step
Archive changes
simulation
Leakage
n/a (generated)not assessed (generated data)
Split sources
Archive default split; who built it and by what rule is not documented RS
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails task construction (generated data)
New case assumed
New generated examples.
Split evidence
generated data: no deployment unit.
Label
generated data. The class semantics and generating equations are undocumented (archive page only) R.
MeatSPECTRO · Spectroscopy & chemistry · 60/60 train/test · length 448 · 3 classesreal tasksource task labelUnclearassumed new case: new physical samplesordered, non-temporalwavelength or wavenumber
Taskreal task
Source task
real task meat species (chicken, pork or turkey)
Archive task
meat species (chicken, pork or turkey)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New physical samples
Exact duplicates
none found D
Why
The source measured each of 60 physical samples twice. We do not know whether the two measurements of a sample were kept together. If one goes into training and the other into testing, the model sees another measurement of a familiar sample. The sample assignment is needed to check whether this happened.
Source
A feasibility study at the Institute of Food Research, Norwich, into using mid-infrared spectroscopy for meat authenticity: can minced chicken, pork and turkey be told apart from their FTIR-ATR spectra?
Publication
Al-Jowder, Kemsley, Wilson, "Mid-infrared spectroscopy and authenticity problems in selected meats: a feasibility study", Food Chemistry 59, 195-201, 1997, doi:10.1016/S0308-8146(96)00289-0. Abstract read via the Europe PMC REST API; full text not reached. The dataset itself is described on the donor's own page, IFR Analytical Sciences "Example Datasets for Download": "120 Mid-infrared (MIR) spectra of fresh minced meats - chicken, pork and turkey. Duplicate acquisitions from 60 independent samples. Raw data matrix size [448 x 120]." That [448 x 120] matches the archive exactly (length 448; 60 train + 60 test).
Full record
Benchmark task
Classify a univariate length-448 mid-IR spectrum as chicken, pork or turkey.
real task human body region depicted in the image (10 classes, unnamed in the archive)
Archive task
human body region depicted in the image (10 classes, unnamed in the archive)
Label attached by
Unknownsource taskSR
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed. The source publication was not located, so this rests on the archive's account.
SplitConcern
New case assumed
New patients or images
Exact duplicates
1 of 760 test series (0.1%) exactly match a training series D
Why
The files do not identify the patient or original image behind each series. Different series might therefore come from one patient or image. We cannot check whether those sources were kept apart. Missing identifiers leave this question unanswered; they do not prove that sources were shared. Separately, the value check reports one test case identical to training data.
Source
Content-based retrieval and classification of medical images by anatomical region. The archive page is the only description: "The data are histograms of pixel intensity of medical images.
Publication
Not identified. Consulted Felipe, Traina & Traina, "Retrieval by Content of Medical Images Using Texture for Tissue Identification", CBMS 2003 (read) and Felipe, Traina & Traina, "A New Family of Distance Functions for Perceptual Similarity Retrieval of Medical Images" (read via PMC), neither of which describes a 10-class, 99-bin intensity-histogram dataset. The archive page gives no data_source_link.
Full record
Benchmark task
Classify a length-99 intensity histogram into one of 10 body-region classes.
Who would use it
hospital PACS systems - automatic organisation and retrieval of stored medical images
Task construction
passnone (source task label)
The benchmark label is the body-region class stated on the archive page (human body region depicted in the image (10 classes, unnamed in the archive)); the source study was not identified, so who attached the labels is unknown R; label origin: source task.
Data origin
ordered, non-temporalindex: histogram bin (intensity histogram)
Index is histogram bin: each series is the intensity histogram of an image; neither physical time nor spatial order is involved.
Archive changes
z-normalization
Leakage
failexact cross-split matches (1 of 760 test series)
Split sources
Archive default split; who built it and by what rule is not documented RDR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails leakage
New case assumed
New patients or images.
Split evidence
nothing documents patient overlap across the split R. The series is a global intensity histogram of the image (index = bin), not a contour; filed as ordered, non-temporal R. Source publication not identified.
Open questions
Index is histogram bin; no reading makes a histogram a time axis. Source publication not identified.
MelbournePedestrianTRAFFIC · Traffic & pedestrians · 1,194/2,439 train/test · length 24 · 10 classesfailsinstance identity as labelRelated periodsassumed new case: new days or later periodstemporaltimefamily: melbourne-pedestrianopen question
Taskfails
Source task
none Pedestrian monitoring; no class target
Archive task
Which of ten sensor locations
Label attached by
Curatorsinstance identityS
Verdict
failsinstance identity as label
The source collected the data but posed no classification target.
SplitRelated periods
New case assumed
New days or later periods
Exact duplicates
none found D
Why
Training and test days come from the same ten city sensors and the same year 2017; the assignment rule is undocumented. Training and test days can share local events, seasons and familiar sensors. The test checks more days from that collection. It does not specifically check a later period or a sensor that was absent from training.
Source
The City of Melbourne runs an automated pedestrian counting system: sensors at fixed sites count people passing and the counts are published hourly as open data, so that the city can understand how different locations are used at different times of day and plan accordingly. The city posed no prediction problem.
Publication
City of Melbourne Pedestrian Counting System open data, read at https://data.melbourne.vic.gov.au/explore/dataset/pedestrian-counting-system-monthly-counts-per-hour/ -- the system "helps to understand how people use different city locations at different times of day to better inform decision-making and plan for the future", and the published dataset contains "hourly pedestrian counts since 2009 from pedestrian sensor devices located across the city". The UCR dataset was cut from the 2017 data by Hoang Anh Dau; no paper describes the curation, only the archive description page.
Full record
Benchmark task
Given a 24-point hourly pedestrian-count profile, identify which of 10 Melbourne sensor locations produced it.
Who would use it
The counts serve city planners and transport analysts; identifying which sensor a trace came from has no stakeholder at all, because the sensor id is part of every record the city publishes.
Task construction
failinstance identity as label
Benchmark target: Which of ten sensor locations. Source target or purpose: Pedestrian monitoring; no class target. Label attached by: Curators. Mechanism 'instance identity as label': the archive uses the recording's source (subject, sensor or actor) as the class.
Data origin
temporalindex: time
Archive changes
windowing
Leakage
pass (related periods)different periods of one ongoing process
Split sources
Days of ten sensors in one year assigned to train/test at random by the archive editor RR
Chinatown is one sensor and year of MelbournePedestrian
Overall verdict
fails task construction
New case assumed
New days or later periods.
Split evidence
days of ten sensors in one year split at random; deployment on those sensors sees new days, but train and test days share the year's events and no chronological split is used R.
Label
sensor-location label taken from the data portal's metadata by the archive editor R. Which sensor location produced the profile is filed as instance identity (the sensor is the source of the series), whereas which freezer, stream or lead is filed as metadata; the boundary is a naming convention of the paper R.
Open questions
Metadata versus identity boundary: which sensor location is filed as instance identity, whereas which freezer, stream or lead (Freezer*, MoteStrain, TwoLeadECG) is filed as metadata. Split: related periods; whether this counts as clean is undecided.
MiddlePhalanxOutlineAgeGroupIMAGE · Medical imaging · 400/154 train/test · length 80 · 3 classesreal tasksource task labelUnclearassumed new case: new patientsordered, non-temporalcontour arc-lengthfamily: phalanx-radiographsopen question
Taskreal task
Source task
real task chronological age of the subject (the study's continuous regression target)
Archive task
age group of the subject (0-6, 7-12, 13-19 years), three curator-defined bands
Label attached by
Curatorssource taskS
Verdict
real task
The source study posed a prediction target (chronological age, modelled by regression) that a stakeholder outside the paper needed; the archive coarsens it into three age groups.
SplitUnclear
New case assumed
New patients
Exact duplicates
none found D
Why
Same as DistalPhalanxOutlineAgeGroup. [DistalPhalanxOutlineAgeGroup: The cases are finger-bone outlines extracted from X-rays. We do not know whether one patient contributed several images or whether those images were kept together. If one patient appears in both sets, the model may benefit from familiar anatomy. Patient identifiers are needed to check performance on new patients.]
Source
All eleven "bone ageing" problems in the archive come from one project: Luke Davis's PhD "Predictive Modelling of Bone Ageing" at UEA, using hand radiographs from the Digital Hand Atlas (Cao et al.).
Publication
Bagnall & Davis, "Predictive Modelling of Bone Age through Classification and Regression of Bone Shapes", Technical Report CMPC14-02, arXiv:1406.4781 (read); images from Cao et al., "Digital hand atlas and web-based bone age assessment: system design and implementation" (not read; cited by the archive page and by Bagnall & Davis).
Full record
Benchmark task
Classify a length-80 middle-phalanx outline series into one of three age bands.
Who would use it
paediatric radiology - bone age assessment is performed daily in hospitals to check whether a child's skeletal development is on track
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (chronological age of the subject, the study's continuous regression target), coarsened by the curators into three age bands; label origin: source task. A coarsened source target still counts as source task.
the same ~1300 hand radiographs of the same subjects; different bones and labels (age band, Tanner-Whitehouse stage, segmentation correctness); PhalangesOutlinesCorrect pools three siblings
Overall verdict
real task; leakage unknown
New case assumed
New patients.
Split evidence
one bone outline per radiograph; nothing states whether a patient's radiographs can appear on both sides R.
Label
the archive page defines three age bands S; they discretise the source's continuous chronological-age target and are treated as a coarsened source target, not a replacement R.
Open questions
Age discretised into three bands by the curators; we treat this as a coarsened source target rather than a replacement.
MiddlePhalanxOutlineCorrectIMAGE · Medical imaging · 600/291 train/test · length 80 · 2 classesreal tasksource task labelUnclearassumed new case: new patientsordered, non-temporalcontour arc-lengthfamily: phalanx-radiographsopen question
Taskreal task
Source task
real task whether the automatic outline extraction for the middle phalanx was judged correct
Archive task
Whether segmentation is judged correct
Label attached by
Source studysource taskS
Verdict
real task
The source study posed outline verification as a stage of its own bone-age pipeline, and the archive kept that correct/incorrect label.
SplitUnclear
New case assumed
New patients
Exact duplicates
none found D
Why
Same as DistalPhalanxOutlineAgeGroup. [DistalPhalanxOutlineAgeGroup: The cases are finger-bone outlines extracted from X-rays. We do not know whether one patient contributed several images or whether those images were kept together. If one patient appears in both sets, the model may benefit from familiar anatomy. Patient identifiers are needed to check performance on new patients.]
Source
All eleven "bone ageing" problems in the archive come from one project: Luke Davis's PhD "Predictive Modelling of Bone Ageing" at UEA, using hand radiographs from the Digital Hand Atlas (Cao et al.).
Publication
Bagnall & Davis, "Predictive Modelling of Bone Age through Classification and Regression of Bone Shapes", Technical Report CMPC14-02, arXiv:1406.4781 (read); images from Cao et al., "Digital hand atlas and web-based bone age assessment: system design and implementation" (not read; cited by the archive page and by Bagnall & Davis).
Full record
Benchmark task
Classify a length-80 middle-phalanx outline series as a correct or incorrect segmentation.
Who would use it
paediatric radiology - bone age assessment is performed daily in hospitals to check whether a child's skeletal development is on track
Task construction
passnone (source task label)
Benchmark target: whether segmentation is judged correct. Source target: the same verification stage of the Bagnall & Davis pipeline ('segment and verify bones'). Label attached by: Source study. Task construction passes.
the same ~1300 hand radiographs of the same subjects; different bones and labels (age band, Tanner-Whitehouse stage, segmentation correctness); PhalangesOutlinesCorrect pools three siblings
Overall verdict
real task; leakage unknown
New case assumed
New patients.
Split evidence
one bone outline per radiograph; nothing states whether a patient's radiographs can appear on both sides R.
Label
as DistalPhalanxOutlineCorrect. The correct/incorrect segmentation label was produced by Davis & Bagnall inside their own bone-age pipeline; it is a property of the segmentation algorithm, not of the patient, and differs from the study's end goal (bone age) S.
Open questions
The evaluators' adjudication rule and agreement, and whether patients were kept apart across train and test, are undocumented.
MiddlePhalanxTWIMAGE · Medical imaging · 399/154 train/test · length 80 · 6 classesreal tasksource task labelUnclearassumed new case: new patientsordered, non-temporalcontour arc-lengthfamily: phalanx-radiographs
Taskreal task
Source task
real task Tanner-Whitehouse skeletal maturity stage of the middle phalanx of the middle finger
Archive task
Tanner-Whitehouse skeletal maturity stage of the middle phalanx of the middle finger
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New patients
Exact duplicates
none found D
Why
Same as DistalPhalanxOutlineAgeGroup. [DistalPhalanxOutlineAgeGroup: The cases are finger-bone outlines extracted from X-rays. We do not know whether one patient contributed several images or whether those images were kept together. If one patient appears in both sets, the model may benefit from familiar anatomy. Patient identifiers are needed to check performance on new patients.]
Source
All eleven "bone ageing" problems in the archive come from one project: Luke Davis's PhD "Predictive Modelling of Bone Ageing" at UEA, using hand radiographs from the Digital Hand Atlas (Cao et al.).
Publication
Bagnall & Davis, "Predictive Modelling of Bone Age through Classification and Regression of Bone Shapes", Technical Report CMPC14-02, arXiv:1406.4781 (read); images from Cao et al., "Digital hand atlas and web-based bone age assessment: system design and implementation" (not read; cited by the archive page and by Bagnall & Davis).
Full record
Benchmark task
Classify a length-80 middle-phalanx outline series into one of 6 Tanner-Whitehouse stages.
Who would use it
paediatric radiology - bone age assessment is performed daily in hospitals to check whether a child's skeletal development is on track
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (Tanner-Whitehouse skeletal maturity stage of the middle phalanx of the middle finger); label origin: source task.
the same ~1300 hand radiographs of the same subjects; different bones and labels (age band, Tanner-Whitehouse stage, segmentation correctness); PhalangesOutlinesCorrect pools three siblings
Overall verdict
real task; leakage unknown
New case assumed
New patients.
Split evidence
one bone outline per radiograph; nothing states whether a patient's radiographs can appear on both sides R.
real task object category depicted by the shape (arrowhead, butterfly, fish, seashell, shield)
Archive task
object category depicted by the shape (arrowhead, butterfly, fish, seashell, shield)
Label attached by
Source studysource taskSR
Verdict
real task
The source study posed automatic annotation of historical images (linking each to a modern counterpart with its meta tags), a task digital libraries indexing scanned collections needed.
SplitConcern
New case assumed
New images or physical objects
Exact duplicates
1 of 2,425 test series (0%) exactly match a training series D
Why
The cases are outlines from shape images. We do not know whether several views or processed versions of one original object can occur in both sets. If they do, the model may recognise a familiar shape. The source-image information needed to check this is not described in the audit. Separately, the value check reports one test case identical to training data.
Source
Automatic annotation of images in digitized historical archives (Wang et al., JCDL 2008): outlines of arrowheads, butterflies, fish, seashells and shields, the kind of object in the source's natural-history plates and artefacts (the source does not itemise this set).
Publication
Wang, Ye, Keogh & Shelton, "Annotating historical archives of images", JCDL 2008 (read; cited as [1] by the archive page), together with Keogh et al., "LB_Keogh supports exact indexing of shapes under rotation invariance with arbitrary representations and distance measures", VLDB 2006 (cited as [2], not read). Neither source itemises this five-class dataset as released, so the composition of the archive version rests on the archive page.
Full record
Benchmark task
Classify a length-1024 outline series into one of five shape classes.
Who would use it
digital libraries wanting to index and annotate image content in scanned historical collections
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (object category depicted by the shape (arrowhead, butterfly, fish, seashell, shield)); label origin: source task.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
failexact cross-split matches (1 of 2425 test series)
Split sources
Dau et al. 2019 (p. 11): one 2425-series test set shared with MixedShapesSmallTrain; 500 training series SD
MixedShapesSmallTrainIMAGE · Shapes & object images · 100/2,425 train/test · length 1024 · 5 classesreal tasksource task labelUnclearassumed new case: new source images or objectsordered, non-temporalcontour arc-lengthfamily: mixed-shapes
Taskreal task
Source task
real task object category depicted by the shape (arrowhead, butterfly, fish, seashell, shield)
Archive task
object category depicted by the shape (arrowhead, butterfly, fish, seashell, shield)
Label attached by
Source studysource taskSR
Verdict
real task
The source study posed automatic annotation of historical images (linking each to a modern counterpart with its meta tags), a task digital libraries indexing scanned collections needed.
SplitUnclear
New case assumed
New source images or objects
Exact duplicates
none found D
Why
Same uncertainty about the original images as MixedShapesRegularTrain. The one value match mentioned there concerns MixedShapesRegularTrain, not this variant. [MixedShapesRegularTrain: The cases are outlines from shape images. We do not know whether several views or processed versions of one original object can occur in both sets. If they do, the model may recognise a familiar shape. The source-image information needed to check this is not described in the audit. Separately, the value check reports one test case identical to training data.]
Source
Automatic annotation of images in digitized historical archives (Wang et al., JCDL 2008): outlines of arrowheads, butterflies, fish, seashells and shields; small-training-set variant sharing the test set of MixedShapesRegularTrain.
Publication
Wang, Ye, Keogh & Shelton, "Annotating historical archives of images", JCDL 2008 (read; cited as [1] by the archive page), together with Keogh et al., "LB_Keogh supports exact indexing of shapes under rotation invariance with arbitrary representations and distance measures", VLDB 2006 (cited as [2], not read). Neither source itemises this five-class dataset as released, so the composition of the archive version rests on the archive page.
Full record
Benchmark task
Classify a length-1024 outline series into one of five shape classes.
Who would use it
digital libraries wanting to index and annotate image content in scanned historical collections
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (object category depicted by the shape (arrowhead, butterfly, fish, seashell, shield)); label origin: source task.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Dau et al. 2019 (p. 11): the same 2425-series test set as MixedShapesRegularTrain; 100 training series SR
none Unsupervised stream correlation tracking; no class target
Archive task
Which source stream: humidity vs temperature
Label attached by
Curatorsinstance identityS
Verdict
failsinstance identity as label
The source collected the data but posed no classification target.
SplitConcern
New case assumed
Independent recordings
Exact duplicates
none found D
Why
The audit traces cases to nearby pieces of a small number of sensor recordings, with the recording defining the class. Training and testing therefore share those original recordings. The model may recognise their particular conditions. The test does not show that the distinction works across recordings collected independently.
Source
Calibrated humidity and temperature streams from a wireless sensor-mote deployment (CMU), used by Sun and Faloutsos as an unsupervised testbed for tracking correlations among co-evolving streams; Keogh later cut windows from two streams and used the stream identity as the class.
Publication
timeseriesclassification.com description page: "Sensor data originally from Carlos Guestrin (CMU) the subsequently used in the paper Online Latent Variable Detection in Sensor Networks and formatted for classification by Eamonn Keogh. The task is to distinguish between sensor q8calibHumid (a humidity measure) and sensor q8calibHumTemp (temperature). The data has dropouts. In the original data the dropouts have value 0. This means they are difficult to detect with normalised data." The linked paper — Sun, Papadimitriou & Faloutsos, "Online Latent Variable Detection in Sensor Networks", ICDE 2005 (demo, pp. 1126-1127) — was read in full; it describes the system and says the demo data source "consists of 8 sensor streams, 4 temperature streams and 4 light intensity streams" from "MICA2 wireless motes with MTS310 sensor boards", and does not mention humidity streams or the q8calib* files. The named files instead appear in the archive's second link (github.com/MrKriss/CDST-demo/tree/master/datasets/Motes), whose README was read: "q8calibHumid.dat - humidity measurements / q8calibHumTemp.dat - temperature measurements ... These datasets are produced by Jimeng Sun and Christos Faloutsos (CMU). The original data are provided by Carlos Guestrin (CMU)."
Full record
Benchmark task
Given a length-84 z-normalized sensor window, decide whether it came from the humidity stream or the temperature stream.
Who would use it
none in the applied sense — nobody needs a model to tell a humidity sensor from a thermometer, since the sensor is known at read time. The realistic use is diagnostic (detecting mislabelled or miswired channels in a sensor network).
Task construction
failinstance identity as label
Benchmark target: Which source stream: humidity vs temperature. Source target or purpose: Unsupervised stream correlation tracking; no class target. Label attached by: Curators. Mechanism 'instance identity as label': each class is one specific sensor stream.
Data origin
temporalindex: time
Archive changes
windowingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Archive default split; who built it and by what rule is not documented RSR
Multichannel source
yes, one channel from a multichannel recording
Dataset family
none from the same source task
Overall verdict
fails task construction and leakage
New case assumed
Independent recordings.
Split evidence
windows of at most two long recordings; train and test windows are neighbouring 84-point segments of the same streams and the class is the stream itself R.
Label
archive page: 'formatted for classification by Eamonn Keogh'; the class is which file the window came from R. The class is which source file (humidity vs temperature stream) a window was cut from; the source study was unsupervised correlation tracking R.
NonInvasiveFetalECGThorax1ECG · Physiological signals · 1,800/1,965 train/test · length 750 · 42 classeslabel meaning undocumentedcounted as a real taskUnclearassumed new case: independent recordingstemporaltimesingle channelfamily: nifecgopen question
Tasklabel meaning undocumented
Source task
undocumented no classification target documented (recordings made for non-invasive fetal ECG extraction)
Archive task
one of 42 classes with no documented meaning
Label attached by
CuratorsundocumentedR
Verdict
label meaning undocumented
No source documents a classification target; the class variable was introduced at donation or by the curators with undocumented meaning.
SplitUnclear
New case assumed
Independent recordings
Exact duplicates
none found D
Why
We read the labels as identifying recordings, with training and test heartbeats coming from those same recordings. However, the source mapping is not confirmed. If correct, testing checks more beats from familiar recordings. This remains a possible concern until the original sources and labels are confirmed.
Source
Thoracic ECG channels recorded alongside abdominal ones for non-invasive fetal ECG extraction (PhysioNet); the archive page does not identify the source and gives 42 classes with no key.
Publication
timeseriesclassification.com description page (which explicitly records the curators' own uncertainty about the source), PhysioNet/CinC Challenge 2013 page linked from it, and PhysioNet Non-Invasive Fetal ECG Database (https://physionet.org/content/nifecgdb/1.0.0/, read 2026-08). No primary paper was identified.
Full record
Benchmark task
Classify a length-750 thoracic-ECG window (thorax 1 channel) into one of 42 unlabelled classes. 1800 train / 1965 test.
Who would use it
obstetricians / non-invasive fetal heart monitoring in pregnancy and labour
Task construction
passnone (label meaning undocumented)
Benchmark target: one of 42 classes with no documented meaning. The source posed no classification target and no source says what the archive's classes mean, so original task and label origin are recorded as undocumented. We do not count it among the constructed tasks, because no source shows that the archive constructed the label; it is counted with the real tasks and flagged as an open question.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded multichannel non-invasive ECG of a pregnant woman; the UCR release keeps a single channel of it. The recordings are multichannel by design and the whole point of the modality is to combine abdominal and thoracic channels to cancel the maternal ECG. This dataset keeps a single thoracic channel (thorax 1) and presents it as a univariate series, so the fetal signal - the reason the data exists - is not recoverable from it at all. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
the two thoracic channels of the same recordings, identical split
Overall verdict
label meaning undocumented (counted with the real tasks); leakage unknown
New case assumed
Independent recordings.
Split evidence
the archive does not identify the source record; if it is nifecgdb the 42 classes are sessions of one woman and every class's beats come from one recording, but that premise is unresolved, so class-internal recording sharing is not established R.
Label
a curator-assigned grouping over recordings whose meaning the archive itself does not know R. The 42 classes have no documented meaning; the reading that they are recording sessions of one subject (nifecgdb) is unverified R. Because the label meaning is undocumented, the dataset is not counted among the constructed tasks; original task and label origin are recorded as undocumented (policy L4).
Open questions
The 42 classes have no documented meaning; if they are sessions of one subject the label would be instance identity. Not counted among the constructed tasks.
NonInvasiveFetalECGThorax2ECG · Physiological signals · 1,800/1,965 train/test · length 750 · 42 classeslabel meaning undocumentedcounted as a real taskUnclearassumed new case: independent source recordingstemporaltimesingle channelfamily: nifecgopen question
Tasklabel meaning undocumented
Source task
undocumented no classification target documented (recordings made for non-invasive fetal ECG extraction)
Archive task
one of 42 classes with no documented meaning
Label attached by
CuratorsundocumentedR
Verdict
label meaning undocumented
No source documents a classification target; the class variable was introduced at donation or by the curators with undocumented meaning.
SplitUnclear
New case assumed
Independent source recordings
Exact duplicates
none found D
Why
Same as NonInvasiveFetalECGThorax1. [NonInvasiveFetalECGThorax1: We read the labels as identifying recordings, with training and test heartbeats coming from those same recordings. However, the source mapping is not confirmed. If correct, testing checks more beats from familiar recordings. This remains a possible concern until the original sources and labels are confirmed.]
Source
Thoracic ECG channels recorded alongside abdominal ones for non-invasive fetal ECG extraction (PhysioNet); the archive page does not identify the source and gives 42 classes with no key.
Publication
timeseriesclassification.com description page (which explicitly records the curators' own uncertainty about the source), PhysioNet/CinC Challenge 2013 page linked from it, and PhysioNet Non-Invasive Fetal ECG Database (https://physionet.org/content/nifecgdb/1.0.0/, read 2026-08). No primary paper was identified.
Full record
Benchmark task
Classify a length-750 thoracic-ECG window (thorax 2 channel) into one of 42 unlabelled classes. 1800 train / 1965 test.
Who would use it
obstetricians / non-invasive fetal heart monitoring in pregnancy and labour
Task construction
passnone (label meaning undocumented)
Benchmark target: one of 42 classes with no documented meaning. The source posed no classification target and no source says what the archive's classes mean, so original task and label origin are recorded as undocumented. We do not count it among the constructed tasks, because no source shows that the archive constructed the label; it is counted with the real tasks and flagged as an open question.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded multichannel non-invasive ECG of a pregnant woman; the UCR release keeps a single channel of it. The recordings are multichannel by design and the whole point of the modality is to combine abdominal and thoracic channels to cancel the maternal ECG. This dataset keeps a single thoracic channel (thorax 2) and presents it as a univariate series, so the fetal signal - the reason the data exists - is not recoverable from it at all. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
the two thoracic channels of the same recordings, identical split
Overall verdict
label meaning undocumented (counted with the real tasks); leakage unknown
New case assumed
Independent source recordings.
Split evidence
the archive does not identify the source record; if it is nifecgdb the 42 classes are sessions of one woman and every class's beats come from one recording, but that premise is unresolved, so class-internal recording sharing is not established R.
Label
as NonInvasiveFetalECGThorax1. The 42 classes have no documented meaning; the reading that they are recording sessions of one subject (nifecgdb) is unverified R. Because the label meaning is undocumented, the dataset is not counted among the constructed tasks; original task and label origin are recorded as undocumented (policy L4).
Open questions
The 42 classes have no documented meaning; if they are sessions of one subject the label would be instance identity. Not counted among the constructed tasks.
OliveOilSPECTRO · Spectroscopy & chemistry · 30/30 train/test · length 570 · 4 classesreal tasksource task labelUnclearassumed new case: new physical samplesordered, non-temporalwavelength or wavenumber
Taskreal task
Source task
real task country of origin of an extra virgin olive oil (4 European countries)
Archive task
country of origin of an extra virgin olive oil (4 European countries)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New physical samples
Exact duplicates
none found D
Why
The source measured 60 oils in duplicate (120 spectra); the archive ships 60 and does not say whether they are one spectrum per oil or both replicates of 30 oils. A nearest-neighbour check weakly favours one per oil but cannot tell which oil produced each series. The test's use of genuinely new samples therefore remains unverified.
Source
A geographic-origin authentication study at the Institute of Food Research, Norwich: can FTIR spectroscopy plus multivariate analysis tell extra virgin olive oils of different producing countries apart? Abstract (J.
Publication
Tapp, Defernez, Kemsley, "FTIR spectroscopy and multivariate analysis can distinguish the geographic origin of extra virgin olive oils", J. Agric. Food Chem. 51(21), 6110-6115, 2003, doi:10.1021/jf030232s. Abstract read via the Europe PMC REST API; full text paywalled. The donor's own page, IFR Analytical Sciences "Example Datasets for Download", describes the release: "120 Mid-infrared (MIR) spectra. Duplicate acquisitions from 60 authenticated extra virgin olive oils from 4 different countries of origin; raw data matrix size [570 x 120]." The 570 matches the archive's series length exactly.
Full record
Benchmark task
Classify a univariate length-570 mid-IR spectrum into one of four countries of origin.
Who would use it
food-authenticity labs and PDO/origin-labelling regulators
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (country of origin of an extra virgin olive oil (4 European countries)); label origin: source task.
Data origin
ordered, non-temporalindex: wavelength or wavenumber
Archive changes
spectrum-as-series
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RDR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New physical samples.
Split evidence
60 spectra of four origins; a nearest-neighbour check weakly favours one spectrum per oil, but replicate structure is undocumented R+D.
real task plant species (four Acer and two Quercus species)
Archive task
plant species (four Acer and two Quercus species)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New physical specimens
Exact duplicates
none found D
Why
The cases are outlines derived from leaf images. The description does not establish how physical specimens and any repeated views were assigned to the two sets. New-specimen evaluation therefore cannot be verified; repeated views are a possibility to check, not an observed overlap.
Source
An Oregon State University MSc project on content-based image retrieval for plant species identification from isolated leaves, already using boundary-angle sequences matched by dynamic programming.
Publication
A. Grandhi, MSc project "Content-Based Image Retrieval: Plant Species Identification", Oregon State University (project website read: index and processing pages at web.engr.oregonstate.edu/~tgd/leaves/; the thesis PDF itself was not retrieved).
Full record
Benchmark task
Classify a length-427 leaf-outline series into one of six species.
Who would use it
botanists and herbaria wanting content-based retrieval/identification over digitized specimen collections
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (plant species (four Acer and two Quercus species)); label origin: source task.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New physical specimens.
Split evidence
one outline per leaf image; split composition undocumented R.
PhalangesOutlinesCorrectIMAGE · Medical imaging · 1,800/858 train/test · length 80 · 2 classesreal tasksource task labelUnclearassumed new case: new x-rays or patientsordered, non-temporalcontour arc-lengthfamily: phalanx-radiographsopen question
Taskreal task
Source task
real task whether the automatic outline extraction for a phalanx was judged correct
Archive task
Union of the three segmentation-correctness tasks
Label attached by
Source studysource taskS
Verdict
real task
The source study posed outline verification as a stage of its own bone-age pipeline, and the archive kept that correct/incorrect label; the pooling of the three bones is the archive's.
SplitUnclear
New case assumed
New X-rays or patients
Exact duplicates
none found D
Why
Several bone outlines can come from one hand X-ray. Unless kept together, one image can supply outlines to both sets, letting the model benefit from familiar image conditions. The audit does not describe such grouping. Image or patient identifiers are needed to establish whether this possible overlap actually occurs.
Source
Part of the same project as the nine phalanx problems: Luke Davis's PhD "Predictive Modelling of Bone Ageing" at UEA, on hand radiographs from the Digital Hand Atlas.
Publication
Bagnall & Davis, "Predictive Modelling of Bone Age through Classification and Regression of Bone Shapes", arXiv:1406.4781 (read) for the pipeline and the outline representation; the pooling of the three correctness problems into one dataset is documented only on the archive page.
Full record
Benchmark task
Classify a length-80 phalanx-outline series (distal, middle or proximal, pooled) as a correct or incorrect segmentation.
Who would use it
the developers of an automated bone-ageing system, who need to reject bad segmentations before the age model sees them (paediatric radiology context)
Task construction
passnone (source task label)
Benchmark target: union of the three segmentation-correctness tasks. Source target: the same verification stage of the Bagnall & Davis pipeline ('segment and verify bones'). Label attached by: Source study. Task construction passes; the pooling of the three bones is the archive's.
the same ~1300 hand radiographs of the same subjects; different bones and labels (age band, Tanner-Whitehouse stage, segmentation correctness); PhalangesOutlinesCorrect pools three siblings
Overall verdict
real task; leakage unknown
New case assumed
New X-rays or patients.
Split evidence
the three bones of one radiograph are pooled, so the same patient's bones sit on both sides of the 1800/858 split R.
Label
as DistalPhalanxOutlineCorrect (the pooling of the three bones is the archive's, the label is the collectors'). The three bones of one radiograph are pooled, so one patient's bones can sit on both sides; inferred from the pooling, not documented R.
Open questions
The evaluators' adjudication rule and agreement, and whether patients were kept apart across train and test, are undocumented.
PhonemeAUDIO · Audio & speech · 214/1,896 train/test · length 1024 · 39 classesreal tasksource task labelUnclearassumed new case: new speakers or spoken recordingstemporaltimefamily: phoneme
Taskreal task
Source task
real task which of 39 English phonemes the audio segment is
Archive task
which of 39 English phonemes the audio segment is
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New speakers or spoken recordings
Exact duplicates
none found D
Why
One spoken word can supply several phoneme segments. We do not know whether complete word recordings or speakers were kept in one set. Segments in both sets could share a voice and recording conditions. The test's use of new speakers or separately recorded words remains unverified.
Source
Large-scale phoneme classification for speech recognition (Hamooni & Mueen, ICDM 2014): 370,000 automatically segmented phoneme waveforms in 39 classes from 78,000 English words; the archive holds a small subsample.
Publication
Hamooni and Mueen, "Dual-domain Hierarchical Classification of Phonetic Time Series", IEEE ICDM 2014, Sec. III Data Collection and Cleaning (read) - "We have collected English words from three sources with their audio files. These consist of 30,000 words from Google Translate, 3,000 words from oxforddictionaries.com and 45000 words from the Merrriam-Webster online dictionary. ... recorded at 22050, 44100 and 11025 samples per second respectively." "After data collection, we segment waveforms of the words to generate phonemes. We use the Forced Aligner tool from the Penn Phonetics Laboratory, which segments a word based on its phonetic spelling in the dictionary." "The segmentation process generated 370,000 phonemes in 39 classes covering the entire dictionary." Sec. III-C - "we do a resampling of the words to have all phoneme waveforms at 22050 sample per seconds."
Full record
Benchmark task
Classify a univariate length-1024 audio segment into 39 phoneme classes.
Who would use it
speech-recognition / pronunciation systems; the authors' motivation is a phoneme classifier that generalizes across recording sources rather than overfitting one corpus
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of 39 English phonemes the audio segment is); label origin: source task.
Data origin
temporalindex: time
Archive changes
windowingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
no
Dataset family
phoneme with PhonemeSpectraUEA
UEA PhonemeSpectra is a multivariate representation of a subset of the same corpus
Overall verdict
real task; leakage unknown
New case assumed
New speakers or spoken recordings.
Split evidence
several phonemes are cut from the same word and speaker; whether segments of one word straddle the split is undocumented R.
PickupGestureWiimoteZHAR · Human motion & gesture · 50/50 train/test · length variable · 10 classesreal tasksource task labelUnclearassumed new case: new recording occasionstemporaltimesingle channelfamily: wiimote-gestures
Taskreal task
Source task
real task which of ten enrolled subjects performed the pick-up (poteg) gesture (biometric identification)
Archive task
which of ten enrolled subjects performed the pick-up (poteg) gesture (biometric identification)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New recording occasions
Exact duplicates
none found D
Why
The labels identify people, with several gesture repetitions per person. We do not know whether those repetitions were recorded on separate occasions or how the occasions were assigned to the sets. The test may check more repetitions from a familiar recording. Identification after recording the person again is not established.
Source
Guna, Humar and Pogacnik proposed identifying a user from the way they handle a control device: a person picks up a Nintendo Wii remote from rest, and the three-axis accelerometer trace of that everyday movement is matched (with DTW) against enrolled templates to log the user in.
Publication
Guna, Humar, Pogacnik, "Intuitive gesture based user identification system", 35th Int. Conf. on Telecommunications and Signal Processing (TSP), IEEE 2012. Full text not read - the paper is closed access (OpenAlex reports oa_status "closed", no repository copy). Its abstract was read in full via the OpenAlex API: "We present an intuitive and very easy to use implicit gesture based identification system that is especially suited for security-wise non-critical applications, such as the user login in the multimedia services. The term 'implicit gesture' in our work refers to a natural physical hand manipulation performed by the user who takes hold of the control device and simply picks it up from its neutral motionless position - a 'pick-up' gesture... 100% recognition accuracy was achieved when using explicit identification signature gestures and over 91% recognition accuracy was achieved when using the implicit 'pick-up' gesture."
Full record
Benchmark task
Given a variable-length z-axis acceleration trace of one pick-up (poteg) gesture, decide which of ten people produced it.
Who would use it
implicit/low-friction user login for non-critical applications such as multimedia services (the source study's stated use case)
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of ten enrolled subjects performed the pick-up (poteg) gesture (biometric identification)); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selection
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded three-axis acceleration from a hand-held Nintendo Wii remote during a single natural handling gesture; the UCR release keeps a single channel of it. Two of three axes dropped; the other nine gestures each subject performed are split off into other data sets, and the archive states that "data are shuffled and randomly sampled so that instances across datasets do not sychronized by dimension or subject", so the per-subject multi-gesture structure that the source system actually used is not recoverable. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
AllGestureWiimoteX/Y/Z are the three acceleration axes of the same recordings; PickupGestureWiimoteZ and ShakeGestureWiimoteZ are the z axis of two gestures from the same 10 subjects, labelled by user
Overall verdict
real task; leakage unknown
New case assumed
New recording occasions.
Split evidence
label = subject; whether the ten repetitions were recorded in more than one session, and whether the split separates sessions, is undocumented R.
The source study posed its own prediction task, which the archive did not keep as the label.
SplitConcern
New case assumed
New recording occasions
Exact duplicates
none found D
Why
The task is to identify pigs. Training uses the first pieces of each pig's before/after pressure recordings; testing uses later pieces of the same recordings (Dau et al. 2019). The same pigs are expected in an identification task. The concern is that the model is tested on familiar recordings, rather than after measuring those pigs again.
Source
Early detection of internal bleeding after surgery, studied in an animal model. Guillame-Bert & Dubrawski (JMLR 2017), Sec. 7.4 "Internal Bleeding Detection Data Set", p. 18: "Undetected internal bleeding during and after surgical procedures is a major medical concern."
Publication
Guillame-Bert, Mathieu, and Artur Dubrawski, "Classification of Time Sequences using Graphs of Temporal Constraints", Journal of Machine Learning Research 18(1), 2017, 4370-4403. PDF read 2026-08 (Sec. 7.4, p. 18; Fig. 8 caption, p. 19).
Full record
Benchmark task
Given an 8-second window of airway pressure from a sedated pig, identify which of 52 animals it came from. 104 train (2 per pig) / 208 test (4 per pig).
Who would use it
Critical-care and trauma clinicians; the goal is an alarm that detects occult haemorrhage minutes rather than tens of minutes after onset.
Task construction
failinstance identity as label
Benchmark target: Which of 52 pigs. Source target or purpose: Before vs during induced bleeding. Label attached by: Curators. Mechanism 'instance identity as label': the archive uses the recording's source (subject, sensor or actor) as the class.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Dau et al. 2019 (p. 10) / archive page: train = the first 2000-point slice, test = the second and third 2000-point slices of each pig's before and after recordings SRS
Multichannel source
yes, one channel from a multichannel recording
The source recorded bedside hemodynamic monitoring of a sedated pig; the UCR release keeps a single channel of it. The source data is multivariate by construction - airway pressure, arterial pressure and central venous pressure were recorded simultaneously from the same animal - and the bleeding signature lives in their joint behaviour. This dataset presents airway pressure alone. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
three simultaneously recorded channels of the same 52-pig experiment, identical classes and split sizes
Overall verdict
fails task construction and leakage
New case assumed
New recording occasions.
Split evidence
the 2 training and 4 test windows of a class are consecutive 8-second slices of the same two 30-second recordings of the same animal R.
Label
the archive authors built the three datasets from the donor's recordings and made each pig a class (Dau et al. 2019, Sec. VI-C4: 'We created three data sets out of this source') S. The 52-way pig-identity label replaced the source study's before/during-bleeding label (Dau et al. 2019: 'Class i is the ith individual pig'); the source authors note that between-animal differences dominate the bleeding contrast S.
The source study posed its own prediction task, which the archive did not keep as the label.
SplitConcern
New case assumed
New sessions of known animals
Exact duplicates
none found D
Why
Same as PigAirwayPressure. [PigAirwayPressure: The task is to identify pigs. Training uses the first pieces of each pig's before/after pressure recordings; testing uses later pieces of the same recordings (Dau et al. 2019). The same pigs are expected in an identification task. The concern is that the model is tested on familiar recordings, rather than after measuring those pigs again.]
Source
Early detection of internal bleeding after surgery, studied in an animal model. Guillame-Bert & Dubrawski (JMLR 2017), Sec. 7.4 "Internal Bleeding Detection Data Set", p. 18: "Undetected internal bleeding during and after surgical procedures is a major medical concern."
Publication
Guillame-Bert, Mathieu, and Artur Dubrawski, "Classification of Time Sequences using Graphs of Temporal Constraints", Journal of Machine Learning Research 18(1), 2017, 4370-4403. PDF read 2026-08 (Sec. 7.4, p. 18; Fig. 8 caption, p. 19).
Full record
Benchmark task
Given an 8-second window of arterial blood pressure from a sedated pig, identify which of 52 animals it came from. 104 train (2 per pig) / 208 test (4 per pig).
Who would use it
Critical-care and trauma clinicians; the goal is an alarm that detects occult haemorrhage minutes rather than tens of minutes after onset.
Task construction
failinstance identity as label
Benchmark target: Which of 52 pigs. Source target or purpose: Before vs during induced bleeding. Label attached by: Curators. Mechanism 'instance identity as label': the archive uses the recording's source (subject, sensor or actor) as the class.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Dau et al. 2019 (p. 10) / archive page: train = the first 2000-point slice, test = the second and third 2000-point slices of each pig's before and after recordings SRS
Multichannel source
yes, one channel from a multichannel recording
The source recorded bedside hemodynamic monitoring of a sedated pig; the UCR release keeps a single channel of it. The source data is multivariate by construction - airway pressure, arterial pressure and central venous pressure were recorded simultaneously from the same animal - and the bleeding signature lives in their joint behaviour. This dataset presents arterial blood pressure alone. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
three simultaneously recorded channels of the same 52-pig experiment, identical classes and split sizes
Overall verdict
fails task construction and leakage
New case assumed
New sessions of known animals.
Split evidence
the 2 training and 4 test windows of a class are consecutive 8-second slices of the same two 30-second recordings of the same animal R.
Label
as PigAirwayPressure. The 52-way pig-identity label replaced the source study's before/during-bleeding label (Dau et al. 2019: 'Class i is the ith individual pig'); the source authors note that between-animal differences dominate the bleeding contrast S.
The source study posed its own prediction task, which the archive did not keep as the label.
SplitConcern
New case assumed
New sessions of known animals
Exact duplicates
none found D
Why
Same as PigAirwayPressure. [PigAirwayPressure: The task is to identify pigs. Training uses the first pieces of each pig's before/after pressure recordings; testing uses later pieces of the same recordings (Dau et al. 2019). The same pigs are expected in an identification task. The concern is that the model is tested on familiar recordings, rather than after measuring those pigs again.]
Source
Early detection of internal bleeding after surgery, studied in an animal model. Guillame-Bert & Dubrawski (JMLR 2017), Sec. 7.4 "Internal Bleeding Detection Data Set", p. 18: "Undetected internal bleeding during and after surgical procedures is a major medical concern."
Publication
Guillame-Bert, Mathieu, and Artur Dubrawski, "Classification of Time Sequences using Graphs of Temporal Constraints", Journal of Machine Learning Research 18(1), 2017, 4370-4403. PDF read 2026-08 (Sec. 7.4, p. 18; Fig. 8 caption, p. 19).
Full record
Benchmark task
Given an 8-second window of central venous pressure from a sedated pig, identify which of 52 animals it came from. 104 train (2 per pig) / 208 test (4 per pig).
Who would use it
Critical-care and trauma clinicians; the goal is an alarm that detects occult haemorrhage minutes rather than tens of minutes after onset.
Task construction
failinstance identity as label
Benchmark target: Which of 52 pigs. Source target or purpose: Before vs during induced bleeding. Label attached by: Curators. Mechanism 'instance identity as label': the archive uses the recording's source (subject, sensor or actor) as the class.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Dau et al. 2019 (p. 10) / archive page: train = the first 2000-point slice, test = the second and third 2000-point slices of each pig's before and after recordings SRS
Multichannel source
yes, one channel from a multichannel recording
The source recorded bedside hemodynamic monitoring of a sedated pig; the UCR release keeps a single channel of it. The source data is multivariate by construction - airway pressure, arterial pressure and central venous pressure were recorded simultaneously from the same animal - and the bleeding signature lives in their joint behaviour. This dataset presents central venous pressure alone. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
three simultaneously recorded channels of the same 52-pig experiment, identical classes and split sizes
Overall verdict
fails task construction and leakage
New case assumed
New sessions of known animals.
Split evidence
the 2 training and 4 test windows of a class are consecutive 8-second slices of the same two 30-second recordings of the same animal R.
Label
as PigAirwayPressure. The 52-way pig-identity label replaced the source study's before/during-bleeding label (Dau et al. 2019: 'Class i is the ith individual pig'); the source authors note that between-animal differences dominate the bleeding contrast S.
real task appliance type (11 classes, e.g. fridge, microwave, hairdryer, heater)
Archive task
appliance type (11 classes, e.g. fridge, microwave, hairdryer, heater)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitConcern
New case assumed
New physical appliances
Exact duplicates
5 of 537 test series (0.9%) exactly match a training series D
Why
The source recorded individual appliances several times. We do not know whether all recordings of an appliance were kept in one set. Familiar appliances might occur in testing, but this is not established by the missing information. Separately, the value check finds five test cases identical to training cases.
Source
PLAID (Plug-Load Appliance Identification Dataset) was collected at Carnegie Mellon to support load identification / non-intrusive load monitoring research.
Publication
Gao, Giri, Kara, Berges, "PLAID: a public dataset of high-resolution electrical appliance measurements for load identification research: demo abstract", BuildSys 2014 (not read -- ACM DL returned 403, and the project site www.plaidplug.com is dead, now a parked domain). Read instead the open-access data descriptor for the same dataset family: Medico, De Baets, Gao, Giri, Kara, Dhaene, Develder, Berges, Deschrijver, "A voltage and current measurement dataset for plug load appliance identification in households", Scientific Data 7, 49 (2020).
Full record
Benchmark task
Classify a variable-length, NaN-padded 30 kHz current waveform (537 train / 537 test) into one of 11 appliance types.
Who would use it
Utilities and demand-side-management / NILM vendors that must infer which appliances are running from electrical measurements; also smart-plug manufacturers.
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (appliance type (11 classes, e.g. fridge, microwave, hairdryer, heater)); label origin: source task.
Data origin
temporalindex: time
Archive changes
windowingchannel-selection
Leakage
failexact cross-split matches (5 of 537 test series)
Split sources
Archive default split; who built it and by what rule is not documented RDIR
Multichannel source
yes, one channel from a multichannel recording
The source recorded high-frequency current and voltage waveforms measured at the plug during an appliance switch-on event; the UCR release keeps a single channel of it. PLAID records both current and voltage; the univariate UCR version keeps a single channel, discarding the voltage reference that power/phase features are computed from. Records are event windows around a switch-on, so the startup transient is over-represented relative to normal operation. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
Dataset family
none from the same source task
Overall verdict
fails leakage
New case assumed
New physical appliances.
Split evidence
three to six records per appliance instance R; the split is not documented as instance-disjoint, so instance sharing is inferred from the absence of a grouping statement rather than observed; a NILM model meets new appliances I. The Concern verdict rests on the five exact cross-split matches D; appliance-instance sharing is inferred from the absence of a grouping statement, not observed I.
Open questions
Concern rests on the exact cross-split matches alone; appliance sharing is not established.
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New physical models or capture sessions
Exact duplicates
none found D
Why
The photographs show die-cast plane models, 30 shape samples per class; the source does not say how many physical models were photographed, nor how models or photography sessions were assigned to the two sets. The test could therefore contain another view of a familiar model; that possibility is not verified from the metadata.
Source
The companion database to Car, from the same computer-vision paper. Thakoor & Gao photographed die-cast replica fighter aircraft from directly above and segmented the silhouettes, in order to test a shape classifier on classes that are deliberately very similar to one another.
Publication
Thakoor & Gao, "Shape Classifier based on Generalized Probabilistic Descent Method with Hidden Markov Descriptor", ICCV 2005, Sec. 7.1 (read in full): "The fighter aeroplane shape database included Mirage, Eurofighter, F-14, Harrier, F-22 and F-15. Since F-14 has two possible shapes, one when its wings are closed and another when its wings are opened, total number of shape classes are seven. Each class includes 30 shape samples. Shape database was created by taking digital pictures of die-cast replica models of these aeroplanes from top. Pictures were captured at 640x480 resolution, and were segmented using Spedge and Medge color image segmentation algorithm. Contours of the segmented planes were used for training and testing of the classifier."
Full record
Benchmark task
Classify a univariate length-144 series (an unrolled aircraft outline) into one of seven aircraft-type classes.
Who would use it
military/aerial surveillance systems classifying aircraft views; in practice a purpose-built vision benchmark
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (fighter aircraft type (Mirage, Eurofighter, F-14 wings-closed, F-14 wings-open, Harrier, F-22, F-15)); label origin: source task.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
both from Thakoor & Gao's silhouette database with the same undocumented outline-to-series mapping
Overall verdict
real task; leakage unknown
New case assumed
New physical models or capture sessions.
Split evidence
30 shape samples per class from photographs of die-cast replica models; the number of physical models and whether views of one model sit on both sides are undocumented R.
PowerConsDEVICE · Energy & appliances · 180/180 train/test · length 144 · 2 classesfailsmetadata as labelConcernduplicates 37.2%temporaltimeopen question
Taskfails
Source task
none None; EDF consumption records
Archive task
Warm vs cold season
Label attached by
CuratorsmetadataSR
Verdict
failsmetadata as label
The source collected the data but posed no classification target.
SplitConcern
New case assumed
New days or later periods
Exact duplicates
67 of 180 test series (37.2%) exactly match a training series D
Why
Nearby days can share weather and routines, and no source says whether a later period was kept for testing. Separately, 67 of 180 test cases exactly match training cases. These matches are a concrete issue beyond the possible relationship between neighbouring days.
Source
The underlying data are individual-household electric power consumption readings attributed by the archive to "EDF R&D, Clamart, France" -- i.e. the research arm of the French utility, which meters households to study consumption and support load forecasting and demand analysis.
Publication
timeseriesclassification.com description page only ("Source: EDF R&D, Clamart, France"; "The sampling rate is every ten-minute over a period of one year"). NO primary publication for PowerCons was located despite targeted searching. The UCI "Individual household electric power consumption" dataset was donated by Georges Hebrail and Alice Berard from the same EDF R&D Clamart group (UCI page read: one-minute readings from a house in Sceaux, December 2006 - November 2010) and is a plausible parent, but the sampling rate and the one-year span do not match, so the link is UNCONFIRMED and must not be treated as provenance.
Full record
Benchmark task
Classify a 144-point daily household load curve as belonging to the warm half (Apr-Sep) or the cold half (Oct-Mar) of the year.
Who would use it
Utility load analysts use household demand curves for forecasting and planning; nobody needs to infer the season from a load curve, since the date is always known -- the task is a curator-built proxy.
Task construction
failmetadata as label
Benchmark target: Warm vs cold season. Source target or purpose: None; EDF consumption records. Label attached by: Unknown. Mechanism 'metadata as label': the archive derives the classes from a recording attribute.
Data origin
temporalindex: time
Archive changes
windowing
Leakage
failexact cross-split matches (67 of 180 test series)
Split sources
Archive default split; who built it and by what rule is not documented RDR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails task construction and leakage
New case assumed
New days or later periods.
Split evidence
days of one household's year, split by an undocumented rule (random or chronological); deployment sees new days of the same meter, adjacent days may sit on both sides R, and 37 % of test days are identical to training days D.
Label
the archive page defines the warm/cold classes by recording month S; it was donated by EDF, the collector, and no source says whether EDF or the archive defined the classes (label-by unknown) R. No primary publication located.
Open questions
No primary publication; label-by unknown (EDF or the archive).
ProximalPhalanxOutlineAgeGroupIMAGE · Medical imaging · 400/205 train/test · length 80 · 3 classesreal tasksource task labelUnclearassumed new case: new patientsordered, non-temporalcontour arc-lengthfamily: phalanx-radiographsopen question
Taskreal task
Source task
real task chronological age of the subject (the study's continuous regression target)
Archive task
age group of the subject (0-6, 7-12, 13-19 years), three curator-defined bands
Label attached by
Curatorssource taskSR
Verdict
real task
The source study posed age prediction from bone outlines (as regression) for paediatric bone-age assessment; the archive coarsened it into three age bands.
SplitUnclear
New case assumed
New patients
Exact duplicates
none found D
Why
Same as DistalPhalanxOutlineAgeGroup. [DistalPhalanxOutlineAgeGroup: The cases are finger-bone outlines extracted from X-rays. We do not know whether one patient contributed several images or whether those images were kept together. If one patient appears in both sets, the model may benefit from familiar anatomy. Patient identifiers are needed to check performance on new patients.]
Source
All eleven "bone ageing" problems in the archive come from one project: Luke Davis's PhD "Predictive Modelling of Bone Ageing" at UEA, using hand radiographs from the Digital Hand Atlas (Cao et al.).
Publication
Bagnall & Davis, "Predictive Modelling of Bone Age through Classification and Regression of Bone Shapes", Technical Report CMPC14-02, arXiv:1406.4781 (read); images from Cao et al., "Digital hand atlas and web-based bone age assessment: system design and implementation" (not read; cited by the archive page and by Bagnall & Davis).
Full record
Benchmark task
Classify a length-80 proximal-phalanx outline series into one of three age bands.
Who would use it
paediatric radiology - bone age assessment is performed daily in hospitals to check whether a child's skeletal development is on track
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (the subject's chronological age, a continuous regression target), discretised by the curators into three bands; label origin: source task. A coarsened source target still counts as source task.
the same ~1300 hand radiographs of the same subjects; different bones and labels (age band, Tanner-Whitehouse stage, segmentation correctness); PhalangesOutlinesCorrect pools three siblings
Overall verdict
real task; leakage unknown
New case assumed
New patients.
Split evidence
one bone outline per radiograph; nothing states whether a patient's radiographs can appear on both sides R.
Label
the archive states the three age bands S; continuous chronological age discretised into three bands by the curators R; treated as a coarsened source target (chronological age), not a replacement R.
Open questions
Age discretised into three bands by the curators; we treat this as a coarsened source target rather than a replacement.
ProximalPhalanxOutlineCorrectIMAGE · Medical imaging · 600/291 train/test · length 80 · 2 classesreal tasksource task labelUnclearassumed new case: new patientsordered, non-temporalcontour arc-lengthfamily: phalanx-radiographsopen question
Taskreal task
Source task
real task Whether the automatic outline extraction of the proximal phalanx was judged correct (verification stage of the bone-age pipeline)
Archive task
Whether segmentation is judged correct
Label attached by
Source studysource taskS
Verdict
real task
The source pipeline's verification stage ('segment and verify bones') posed this correct/incorrect target, and the archive kept it as the label.
SplitUnclear
New case assumed
New patients
Exact duplicates
none found D
Why
Same as DistalPhalanxOutlineAgeGroup. [DistalPhalanxOutlineAgeGroup: The cases are finger-bone outlines extracted from X-rays. We do not know whether one patient contributed several images or whether those images were kept together. If one patient appears in both sets, the model may benefit from familiar anatomy. Patient identifiers are needed to check performance on new patients.]
Source
All eleven "bone ageing" problems in the archive come from one project: Luke Davis's PhD "Predictive Modelling of Bone Ageing" at UEA, using hand radiographs from the Digital Hand Atlas (Cao et al.).
Publication
Bagnall & Davis, "Predictive Modelling of Bone Age through Classification and Regression of Bone Shapes", Technical Report CMPC14-02, arXiv:1406.4781 (read); images from Cao et al., "Digital hand atlas and web-based bone age assessment: system design and implementation" (not read; cited by the archive page and by Bagnall & Davis).
Full record
Benchmark task
Classify a length-80 proximal-phalanx outline series as a correct or incorrect segmentation.
Who would use it
paediatric radiology - bone age assessment is performed daily in hospitals to check whether a child's skeletal development is on track
Task construction
passnone (source task label)
Benchmark target: Whether segmentation is judged correct. Source target: the same verification judgement, posed by the source pipeline's 'segment and verify bones' stage. Label attached by: Source study (three human evaluators in Davis's project). No construction mechanism.
the same ~1300 hand radiographs of the same subjects; different bones and labels (age band, Tanner-Whitehouse stage, segmentation correctness); PhalangesOutlinesCorrect pools three siblings
Overall verdict
real task; leakage unknown
New case assumed
New patients.
Split evidence
one bone outline per radiograph; nothing states whether a patient's radiographs can appear on both sides R.
Label
as DistalPhalanxOutlineCorrect. The correct/incorrect segmentation label was produced by Davis & Bagnall inside their own bone-age pipeline; it is a property of the segmentation algorithm, not of the patient, and differs from the study's end goal (bone age) S.
Open questions
The evaluators' adjudication rule and agreement, and whether patients were kept apart across train and test, are undocumented.
ProximalPhalanxTWIMAGE · Medical imaging · 400/205 train/test · length 80 · 6 classesreal tasksource task labelUnclearassumed new case: new patientsordered, non-temporalcontour arc-lengthfamily: phalanx-radiographs
Taskreal task
Source task
real task Tanner-Whitehouse skeletal maturity stage of the proximal phalanx of the middle finger
Archive task
Tanner-Whitehouse skeletal maturity stage of the proximal phalanx of the middle finger
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New patients
Exact duplicates
none found D
Why
Same as DistalPhalanxOutlineAgeGroup. [DistalPhalanxOutlineAgeGroup: The cases are finger-bone outlines extracted from X-rays. We do not know whether one patient contributed several images or whether those images were kept together. If one patient appears in both sets, the model may benefit from familiar anatomy. Patient identifiers are needed to check performance on new patients.]
Source
All eleven "bone ageing" problems in the archive come from one project: Luke Davis's PhD "Predictive Modelling of Bone Ageing" at UEA, using hand radiographs from the Digital Hand Atlas (Cao et al.).
Publication
Bagnall & Davis, "Predictive Modelling of Bone Age through Classification and Regression of Bone Shapes", Technical Report CMPC14-02, arXiv:1406.4781 (read); images from Cao et al., "Digital hand atlas and web-based bone age assessment: system design and implementation" (not read; cited by the archive page and by Bagnall & Davis).
Full record
Benchmark task
Classify a length-80 proximal-phalanx outline series into one of 6 Tanner-Whitehouse stages.
Who would use it
paediatric radiology - bone age assessment is performed daily in hospitals to check whether a child's skeletal development is on track
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (Tanner-Whitehouse skeletal maturity stage of the proximal phalanx of the middle finger); label origin: source task.
the same ~1300 hand radiographs of the same subjects; different bones and labels (age band, Tanner-Whitehouse stage, segmentation correctness); PhalangesOutlinesCorrect pools three siblings
Overall verdict
real task; leakage unknown
New case assumed
New patients.
Split evidence
one bone outline per radiograph; nothing states whether a patient's radiographs can appear on both sides R.
RefrigerationDevicesDEVICE · Energy & appliances · 375/375 train/test · length 720 · 3 classesreal tasksource task labelUnclearassumed new case: new appliances or householdstemporaltimefamily: household-electricity-surveyopen question
Taskreal task
Source task
real task Household electricity monitoring for energy policy (Household Electricity Survey); each metered channel is annotated with its appliance, but the survey posed no classification task
Archive task
which refrigeration appliance produced the day's consumption trace (fridge/freezer, refrigerator, upright freezer)
Label attached by
Collectorssource taskSR
Verdict
real task
The survey's installing electricians annotated each monitored product with its appliance ("selected from a dropdown list"), so under policy L3 we count appliance-type recognition as the source's own task although the survey trained no classifier; the classification problem was posed on those annotations by the curators (Lines & Bagnall, SDM 2014; Lines 2015 thesis). Each input is an appliance trace metered on its own, so the task is appliance recognition, not whole-house disaggregation.
SplitUnclear
New case assumed
New appliances or households
Exact duplicates
none found D
Why
Same as Computers. [Computers: The source measured each physical appliance over many days. We do not know whether all days from an appliance or household were kept in one set. If the same appliance appears in both sets, the model may recognise its usual pattern. That would be a narrower test than recognising appliance types in new homes.]
Source
These series come from the UK government-sponsored household electricity monitoring study known as "Powering the Nation" (the DECC/Defra/Energy Saving Trust Household Electricity Survey).
Publication
UK Household Electricity Survey (DECC, Defra and the Energy Saving Trust, 2010-2011); Intertek Report R66141, "Household Electricity Survey: A study of domestic electrical product usage", final report issue 4. Household counts differ: the archive page, the thesis and Intertek (p. 24) give 251 households, Report 5 (p. 4) and gov.uk give 250; the 26 year-long households were sampled every 10 minutes except from late November 2010 to February 2011, when they recorded every 2 minutes (Intertek p. 64). Curated into TSC datasets by Lines & Bagnall (UEA): J. Lines, PhD thesis, UEA, 2015, Sec. 3.2.2 and Table 3.3 ("We provide five new classification problems that focus on domestic appliance classification from electricity consumption"); introduced in Lines & Bagnall, "Ensembles of elastic distance measures for time series classification", SIAM SDM 2014 (not read: no full text found).
Full record
Benchmark task
Classify a 720-point daily consumption profile as fridge/freezer, refrigerator or upright freezer.
Who would use it
UK energy policy (the survey); the curators' stated aim was to extract more from smart-meter data by "classifying appliances according to their electricity usage profiles". The derived task (naming a device from its own metered daily profile) is appliance recognition, not disaggregation of a whole-house load.
Task construction
passnone (source task label)
The benchmark label is the survey's appliance annotation of the metered channel (which refrigeration appliance produced the day's consumption trace (fridge/freezer, refrigerator, upright freezer)); label origin: source task, by the convention that a collectors' annotation of the quantity a deployed model would predict counts as the source task.
Data origin
temporalindex: time
Archive changes
windowingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split, made by the curators (Lines & Bagnall), who say these datasets were "split in a certain manner to avoid introducing bias" from one device appearing in both train and test S; the rule used is not stated and the files carry no household or device identifiers RIR
five appliance-group tasks cut from the UK Household Electricity Survey (ElectricDevices comes from a different study, the Visual Energy Trial); whether they share households or appliances is not documented
Overall verdict
real task; leakage unknown
New case assumed
New appliances or households.
Split evidence
each series is one day cut from a month (or, for some households, a year) of metering, so one appliance can supply many series R; which households, appliances and days entered the dataset is not documented. The curators say the split was made to avoid device bias, but give no rule, and the files carry no device identifiers, so device- or household-disjointness is neither documented nor refuted and sharing is not counted as leakage R. An appliance-recognition model meeting unseen appliances is the deployment case I.
Open questions
Appliance-type recognition is counted as the source task by convention (the survey annotated the channels but posed no classification task); a stricter reading files it as metadata. Appliance/household sharing inferred from the survey design but not documented, so Unclear rather than Concern.
RockSPECTRO · Earth & space science · 20/50 train/test · length 2844 · 4 classesfailsmetadata as labelUnclearassumed new case: new physical samplesordered, non-temporalwavelength or wavenumber
Taskfails
Source task
none ASTER spectral-library metadata; no defined task
Archive task
Four rock types
Label attached by
CuratorsmetadataSR
Verdict
failsmetadata as label
The source collected the data but posed no classification target.
SplitUnclear
New case assumed
New physical samples
Exact duplicates
none found D
Why
Our similarity check found strongly similar pairs of sequences. That alone does not show that the same physical rock was measured twice. We need sample identifiers to know whether the two sets share rocks. Until then, repeated measurements are a possibility, rather than confirmed overlap.
Source
There was no original prediction problem. The data come from the ASTER Spectral Library v2.0, a NASA/JPL reference collection built to support interpretation of ASTER satellite imagery.
Publication
Baldridge, Hook, Grove, Rivera, "The ASTER spectral library version 2.0", Remote Sensing of Environment 113(4), 711-715, 2009, doi:10.1016/j.rse.2008.11.007. Read in full (open PDF at speclib.jpl.nasa.gov/downloads/2009-Baldridge.pdf). It documents the spectra; it does not document the UCR extract - which 70 samples were taken, how the 20/50 split was made, and how the 2844 points relate to the library's two acquisition ranges (0.4-2.5 um and 2-15.4 um) are nowhere stated.
Full record
Benchmark task
Classify a univariate length-2844 reflectance spectrum as mafic, quartzite, marble or schist.
Who would use it
geologists and remote-sensing analysts use the library as a reference for mineral mapping; no one posed this 70-sample supervised classification
Task construction
failmetadata as label
Benchmark target: Four rock types. Source target or purpose: ASTER spectral-library metadata; no defined task. Label attached by: Curators. Mechanism 'metadata as label': the archive turns the library's rock-type metadata (Subclass field) into the class, for a source that posed no classification.
Data origin
ordered, non-temporalindex: wavelength or wavenumber
Archive changes
spectrum-as-series
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RDR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails task construction (leakage unknown)
New case assumed
New physical samples.
Split evidence
near-duplicate pairs cross the split (r > 0.99) and the test set holds one exact duplicate pair; replicate leakage is plausible but unproven for near-single-mineral rocks R+D.
Label
four rock classes chosen by the archive contributor (Y. Zhu) from the ASTER spectral library's metadata; the library's authors posed no classification R. The four classes were chosen by the archive contributor (Y. Zhu, a UCR archive co-author) from ASTER library metadata; filed as Curators R.
ScreenTypeDEVICE · Energy & appliances · 375/375 train/test · length 720 · 3 classesreal tasksource task labelUnclearassumed new case: new appliances or householdstemporaltimefamily: household-electricity-surveyopen question
Taskreal task
Source task
real task Household electricity monitoring for energy policy (Household Electricity Survey); each metered channel is annotated with its appliance, but the survey posed no classification task
Archive task
which display device produced the day's consumption trace (CRT TV, LCD TV, computer monitor)
Label attached by
Collectorssource taskSR
Verdict
real task
The survey's installing electricians annotated each monitored product with its appliance ("selected from a dropdown list"), so under policy L3 we count appliance-type recognition as the source's own task although the survey trained no classifier; the classification problem was posed on those annotations by the curators (Lines & Bagnall, SDM 2014; Lines 2015 thesis). Each input is an appliance trace metered on its own, so the task is appliance recognition, not whole-house disaggregation.
SplitUnclear
New case assumed
New appliances or households
Exact duplicates
none found D
Why
Same as Computers. [Computers: The source measured each physical appliance over many days. We do not know whether all days from an appliance or household were kept in one set. If the same appliance appears in both sets, the model may recognise its usual pattern. That would be a narrower test than recognising appliance types in new homes.]
Source
These series come from the UK government-sponsored household electricity monitoring study known as "Powering the Nation" (the DECC/Defra/Energy Saving Trust Household Electricity Survey).
Publication
UK Household Electricity Survey (DECC, Defra and the Energy Saving Trust, 2010-2011); Intertek Report R66141, "Household Electricity Survey: A study of domestic electrical product usage", final report issue 4. Household counts differ: the archive page, the thesis and Intertek (p. 24) give 251 households, Report 5 (p. 4) and gov.uk give 250; the 26 year-long households were sampled every 10 minutes except from late November 2010 to February 2011, when they recorded every 2 minutes (Intertek p. 64). Curated into TSC datasets by Lines & Bagnall (UEA): J. Lines, PhD thesis, UEA, 2015, Sec. 3.2.2 and Table 3.3 ("We provide five new classification problems that focus on domestic appliance classification from electricity consumption"); introduced in Lines & Bagnall, "Ensembles of elastic distance measures for time series classification", SIAM SDM 2014 (not read: no full text found).
Full record
Benchmark task
Classify a 720-point daily consumption profile as CRT TV, LCD TV or computer monitor.
Who would use it
UK energy policy (the survey); the curators' stated aim was to extract more from smart-meter data by "classifying appliances according to their electricity usage profiles". The derived task (naming a device from its own metered daily profile) is appliance recognition, not disaggregation of a whole-house load.
Task construction
passnone (source task label)
The benchmark label is the survey's appliance annotation of the metered channel (which display device produced the day's consumption trace (CRT TV, LCD TV, computer monitor)); label origin: source task, by the convention that a collectors' annotation of the quantity a deployed model would predict counts as the source task.
Data origin
temporalindex: time
Archive changes
windowingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split, made by the curators (Lines & Bagnall), who say these datasets were "split in a certain manner to avoid introducing bias" from one device appearing in both train and test S; the rule used is not stated and the files carry no household or device identifiers RIR
five appliance-group tasks cut from the UK Household Electricity Survey (ElectricDevices comes from a different study, the Visual Energy Trial); whether they share households or appliances is not documented
Overall verdict
real task; leakage unknown
New case assumed
New appliances or households.
Split evidence
each series is one day cut from a month (or, for some households, a year) of metering, so one appliance can supply many series R; which households, appliances and days entered the dataset is not documented. The curators say the split was made to avoid device bias, but give no rule, and the files carry no device identifiers, so device- or household-disjointness is neither documented nor refuted and sharing is not counted as leakage R. An appliance-recognition model meeting unseen appliances is the deployment case I.
Open questions
Appliance-type recognition is counted as the source task by convention (the survey annotated the channels but posed no classification task); a stricter reading files it as metadata. Appliance/household sharing inferred from the survey design but not documented, so Unclear rather than Concern.
The source study posed its own prediction task, which the archive did not keep as the label.
SplitConcern
New case assumed
New people
Exact duplicates
none found D
Why
We found the same five people in both sets, checked using labels from the related person-identification dataset. Recognising a familiar person could help predict their gender label. The test checks further measurements of these people, rather than establishing performance on people absent from training.
Source
Surface EMG of the forearm recorded by Sapsanis et al. (2013) from five healthy subjects performing six everyday grasps, to build myoelectric grasp recognition for prosthetic or assistive hands; the archive replaces each repetition by its power spectrum.
Publication
Sapsanis, Georgoulas, Tzes, Lymberopoulos, "Improving EMG based classification of basic hand movements using EMD", IEEE EMBC 2013 - not read (IEEE Xplore, paywalled); only its title and the archive's citation of it. The recording protocol above is quoted from the UCI Machine Learning Repository page for the dataset, which was read. The archive's own processing step ("We extract the power spectrum from the raw data") is stated on the timeseriesclassification.com page and in the README shipped inside the dataset zip, but is documented nowhere in more detail.
Full record
Benchmark task
Classify a univariate length-1500 sEMG power spectrum as produced by a female or a male subject.
Who would use it
prosthetics and assistive-device researchers building myoelectric grasp control; rehabilitation and HCI groups
Task construction
failmetadata as label
Benchmark target: Sex of five subjects. Source target or purpose: Six hand-grasp types. Label attached by: Curators. Mechanism 'metadata as label': the archive derives the classes from a recording attribute.
Data origin
ordered, non-temporalindex: frequency (power spectrum)
Index is frequency: the archive replaces each six-second EMG repetition by its power spectrum, so the ordered axis is spectral, not time.
Archive changes
channel-selectionspectrum-as-series
Leakage
faildocumented or data-verified shared source
Split sources
Undocumented 300/600 split of the 900 series, unequal per subject (train 57/41/52/77/73, test 123/139/128/103/107) D; one recording session per subject RDR
Multichannel source
yes, one channel from a multichannel recording
The source recorded surface EMG recording from two forearm electrodes during a hand grasp; the UCR release keeps a single channel of it. The Fourier transform discards all phase and all temporal structure: burst onset, contraction timing, the ramp of the grasp, and every relation between the two forearm channels (only one is kept). What remains is an order-free energy-by-frequency profile - the archive did the feature extraction that an EMG practitioner would have done, and then presented the features as a "time series". The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
the same 900 series (byte-identical), labelled by grasp, by subject or by sex
Overall verdict
fails task construction and leakage
New case assumed
New people.
Split evidence
five subjects and every subject appears on both sides (verified by joining the SemgHandSubjectCh2 labels by series hash); a sex classifier is deployed on new people R+D.
Label
sex label attached by the archive editor (C. Yeh) from subject metadata; the source study's target was the grasp R. The 1500-point series is the power spectrum of one 6-second grasp repetition (index = frequency); filed as ordered, non-temporal. Subject sharing across the split was verified by joining the SemgHandSubjectCh2 labels by series hash D.
Open questions
Index is frequency; a reader who regards a power spectrum as a time axis would count it as temporal.
real task which of six hand grasps was performed (cylindrical, hook, tip, palmar, spherical, lateral)
Archive task
which of six hand grasps was performed (cylindrical, hook, tip, palmar, spherical, lateral)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitConcern
New case assumed
New people or recording occasions
Exact duplicates
none found D
Why
The audit reports repeated grasps by the same people in the same recording sessions across both sets. These repetitions share the person and electrode placement. The test does not show whether recognition works for a new person or after removing and attaching the electrodes again. Those are separate evaluation settings.
Source
Surface EMG of the forearm recorded by Sapsanis et al. (2013) from five healthy subjects performing six everyday grasps, to build myoelectric grasp recognition for prosthetic or assistive hands; the archive replaces each repetition by its power spectrum.
Publication
Sapsanis, Georgoulas, Tzes, Lymberopoulos, "Improving EMG based classification of basic hand movements using EMD", IEEE EMBC 2013 - not read (IEEE Xplore, paywalled); only its title and the archive's citation of it. The recording protocol above is quoted from the UCI Machine Learning Repository page for the dataset, which was read. The archive's own processing step ("We extract the power spectrum from the raw data") is stated on the timeseriesclassification.com page and in the README shipped inside the dataset zip, but is documented nowhere in more detail.
Full record
Benchmark task
Classify a univariate length-1500 sEMG power spectrum into one of six hand grasps.
Who would use it
prosthetics and assistive-device researchers building myoelectric grasp control; rehabilitation and HCI groups
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of six hand grasps was performed (cylindrical, hook, tip, palmar, spherical, lateral)); label origin: source task.
Data origin
ordered, non-temporalindex: frequency (power spectrum)
Index is frequency: the archive replaces each six-second EMG repetition by its power spectrum, so the ordered axis is spectral, not time.
Archive changes
channel-selectionspectrum-as-series
Leakage
faildocumented or data-verified shared source
Split sources
Undocumented 450/450 split of the 900 series (not defined by the source); per subject 90/77/93/97/93 train and 90/103/87/83/87 test, every subject on both sides D; one recording session per subject RD
Multichannel source
yes, one channel from a multichannel recording
The source recorded surface EMG recording from two forearm electrodes during a hand grasp; the UCR release keeps a single channel of it. The Fourier transform discards all phase and all temporal structure: burst onset, contraction timing, the ramp of the grasp, and every relation between the two forearm channels (only one is kept). What remains is an order-free energy-by-frequency profile - the archive did the feature extraction that an EMG practitioner would have done, and then presented the features as a "time series". The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
the same 900 series (byte-identical), labelled by grasp, by subject or by sex
Overall verdict
fails leakage
New case assumed
New people or recording occasions.
Split evidence
every subject appears on both sides, 90 train / 90 test per subject, verified by joining the SemgHandSubjectCh2 labels by series hash D; the repetitions come from one session per subject, and prosthetic control must generalise across sessions and subjects R. The 1500-point series is the power spectrum of one 6-second grasp repetition (index = frequency); filed as ordered, non-temporal. Session-specific cues (electrode placement, skin impedance, mains coupling) are shared by all repetitions of a subject R.
Open questions
Index is frequency; a reader who regards a power spectrum as a time axis would count it as temporal.
The source study posed its own prediction task, which the archive did not keep as the label.
SplitConcern
New case assumed
New recording occasions
Exact duplicates
none found D
Why
The task identifies people, but each person was recorded on one occasion according to the audit. Both sets contain repetitions from that occasion. The model might recognise that recording setup along with the person. The test does not establish identification after measuring the person again on another occasion.
Source
Surface EMG of the forearm recorded by Sapsanis et al. (2013) from five healthy subjects performing six everyday grasps, to build myoelectric grasp recognition for prosthetic or assistive hands; the archive replaces each repetition by its power spectrum.
Publication
Sapsanis, Georgoulas, Tzes, Lymberopoulos, "Improving EMG based classification of basic hand movements using EMD", IEEE EMBC 2013 - not read (IEEE Xplore, paywalled); only its title and the archive's citation of it. The recording protocol above is quoted from the UCI Machine Learning Repository page for the dataset, which was read. The archive's own processing step ("We extract the power spectrum from the raw data") is stated on the timeseriesclassification.com page and in the README shipped inside the dataset zip, but is documented nowhere in more detail.
Full record
Benchmark task
Classify a univariate length-1500 sEMG power spectrum by which of five subjects produced it.
Who would use it
prosthetics and assistive-device researchers building myoelectric grasp control; rehabilitation and HCI groups
Task construction
failinstance identity as label
Benchmark target: Which of five subjects. Source target or purpose: Six hand-grasp types. Label attached by: Curators. Mechanism 'instance identity as label': the archive uses the recording's source (subject, sensor or actor) as the class.
Data origin
ordered, non-temporalindex: frequency (power spectrum)
Index is frequency: the archive replaces each six-second EMG repetition by its power spectrum, so the ordered axis is spectral, not time.
Archive changes
channel-selectionspectrum-as-series
Leakage
faildocumented or data-verified shared source
Split sources
Stratified shuffle of the 900 series: 90 train / 90 test per subject, one recording session per subject RSR
Multichannel source
yes, one channel from a multichannel recording
The source recorded surface EMG recording from two forearm electrodes during a hand grasp; the UCR release keeps a single channel of it. The Fourier transform discards all phase and all temporal structure: burst onset, contraction timing, the ramp of the grasp, and every relation between the two forearm channels (only one is kept). What remains is an order-free energy-by-frequency profile - the archive did the feature extraction that an EMG practitioner would have done, and then presented the features as a "time series". The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
the same 900 series (byte-identical), labelled by grasp, by subject or by sex
Overall verdict
fails task construction and leakage
New case assumed
New recording occasions.
Split evidence
the archive page defines the classes as the five subjects, so every subject is on both sides S; each subject was recorded once, so the split is a stratified shuffle of one session per subject and the test presents no new session of a known subject R.
Label
subject-identity label attached by the archive editor, stated on the archive page S; not posed in the source study R. The 1500-point series is the power spectrum of one 6-second grasp repetition (index = frequency); filed as ordered, non-temporal.
Open questions
Index is frequency; a reader who regards a power spectrum as a time axis would count it as temporal.
ShakeGestureWiimoteZHAR · Human motion & gesture · 50/50 train/test · length variable · 10 classesreal tasksource task labelUnclearassumed new case: new sessions of known userstemporaltimesingle channelfamily: wiimote-gestures
Taskreal task
Source task
real task which of ten enrolled subjects performed the shake gesture (biometric identification)
Archive task
which of ten enrolled subjects performed the shake gesture (biometric identification)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New sessions of known users
Exact duplicates
none found D
Why
Same as PickupGestureWiimoteZ. [PickupGestureWiimoteZ: The labels identify people, with several gesture repetitions per person. We do not know whether those repetitions were recorded on separate occasions or how the occasions were assigned to the sets. The test may check more repetitions from a familiar recording. Identification after recording the person again is not established.]
Source
Guna, Humar and Pogacnik proposed identifying a user from the way they handle a control device: a person picks up a Nintendo Wii remote from rest, and the three-axis accelerometer trace of that everyday movement is matched (with DTW) against enrolled templates to log the user in.
Publication
Guna, Humar, Pogacnik, "Intuitive gesture based user identification system", 35th Int. Conf. on Telecommunications and Signal Processing (TSP), IEEE 2012. Full text not read - the paper is closed access (OpenAlex reports oa_status "closed", no repository copy). Its abstract was read in full via the OpenAlex API: "We present an intuitive and very easy to use implicit gesture based identification system that is especially suited for security-wise non-critical applications, such as the user login in the multimedia services. The term 'implicit gesture' in our work refers to a natural physical hand manipulation performed by the user who takes hold of the control device and simply picks it up from its neutral motionless position - a 'pick-up' gesture... 100% recognition accuracy was achieved when using explicit identification signature gestures and over 91% recognition accuracy was achieved when using the implicit 'pick-up' gesture."
Full record
Benchmark task
Given a variable-length z-axis acceleration trace of one shake gesture, decide which of ten people produced it.
Who would use it
implicit/low-friction user login for non-critical applications such as multimedia services (the source study's stated use case)
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of ten enrolled subjects performed the shake gesture (biometric identification)); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selection
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded three-axis acceleration from a hand-held Nintendo Wii remote during a single natural handling gesture; the UCR release keeps a single channel of it. Two of three axes dropped; the other nine gestures each subject performed are split off into other data sets, and the archive states that "data are shuffled and randomly sampled so that instances across datasets do not sychronized by dimension or subject", so the per-subject multi-gesture structure that the source system actually used is not recoverable. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
AllGestureWiimoteX/Y/Z are the three acceleration axes of the same recordings; PickupGestureWiimoteZ and ShakeGestureWiimoteZ are the z axis of two gestures from the same 10 subjects, labelled by user
Overall verdict
real task; leakage unknown
New case assumed
New sessions of known users.
Split evidence
label = subject; whether the ten repetitions were recorded in more than one session, and whether the split separates sessions, is undocumented R.
ShapeletSimSIMULATED · Simulated · 20/180 train/test · length 500 · 2 classesfailstoy source taskGeneratedassumed new case: new generated examplessyntheticgenerator step
Taskfails
Source task
toy task None; simulator built so that shapelet methods win
Archive task
Which of two shapes is embedded in noise
Label attached by
Generatorsource taskS
Verdict
failstoy source task
The source data were staged or generated to demonstrate a method.
SplitGenerated
New case assumed
New generated examples
Exact duplicates
none found D
Why
This is a generated task rather than measurements from people, devices or specimens. The corresponding source-separation question is therefore not assessed here. Its split would instead need to be examined through the generation procedure; synthetic origin alone is not evidence of leakage.
Source
Generated by the UEA group (Hills, Bagnall) as a probe for shapelet algorithms: a class is defined by one short discriminative shape inserted at a random position in white noise; no real-world problem exists.
Publication
Bagnall, Lines, Vickers et al., "Simulated Data Experiments for Time Series Classification Part 1 - Accuracy Comparison with Default Settings", arXiv:1703.09480, Sec. 5 "Shapelet Simulator" (read) - "A shapelet is a discriminatory pattern embedded in a series. Currently this simulator only works for a single shapelet defining a class. Each model randomly chooses a shapelet from those described in Figure ??. The shapelet is placed at a random location for each series from that model (by the call to generateSeries(int length)), embedded in noise." The design intent is stated in Sec. 5.1 - "we think vector based classifiers would be poor at these data, because the random location of the discriminatory shape should confound them ... shapelet classifiers should be the best". The archive page names the same generator class (SimulateShapeletDataset) and the shape library (TRIANGLE, HEADSHOULDERS, SINE, STEP, SPIKE). Hills, Lines, Baranauskas, Mapp, Bagnall, "Classification of time series by shapelet transformation", DMKD 28(4), 2014 - the paper the donors are known for - was not reached (the CORE mirror returned HTTP 522, the Springer copy is paywalled), so the parameters used for the archived copy in particular are undocumented.
Full record
Benchmark task
Classify a univariate length-500 series by which embedded shape it contains (2 classes).
Who would use it
none (synthetic demo)
Task construction
failtoy source task
Benchmark target: Which of two shapes is embedded in noise. Source target or purpose: None; simulator built so that shapelet methods win. Label attached by: Generator. Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question.
Data origin
syntheticindex: generator step
Archive changes
simulation
Leakage
n/a (generated)not assessed (generated data)
Split sources
Archive default split; who built it and by what rule is not documented RS
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails task construction (generated data)
New case assumed
New generated examples.
Split evidence
generated data: no deployment unit.
Label
generated data. Simulator built so that shapelet methods separate the classes (one of two shapes embedded at a random position in noise) S.
real task shape retrieval over 70 object classes (MPEG-7 CE-Shape-1 bulls-eye test); no applied problem
Archive task
shape class (60 object categories)
Label attached by
Source studysource taskS
Verdict
real task
The source posed shape retrieval over its 70 MPEG-7 classes, scored by class membership, to compare shape descriptors; it has no applied stakeholder. Counted as a real task under the MPEG-7 policy, as for Car and Plane.
SplitConcern
New case assumed
New source images or objects
Exact duplicates
2 of 600 test series (0.3%) exactly match a training series D
Why
The cases are outlines derived from shape images. The documentation does not establish whether repeated views or versions of one source object were kept in one set. Independence at the original image or object level remains unknown; no particular shared object is verified here. Separately, the value check reports two test cases identical to training data.
Source
The images come from MPEG-7 Core Experiment CE-Shape-1 Part B, a database built to benchmark 2D shape descriptors for the MPEG-7 standard rather than to solve an applied problem. Latecki et al. (CVPR 2000, Sec. 2.2): "The total number of images in the database is 1400: 70 classes of various shapes, each class with 20 images.
Publication
Latecki, Lakamper & Eckhardt, "Shape Descriptors for Non-rigid Shapes with a Single Closed Contour", CVPR 2000 (read). The outline-to-series mapping is described only on the archive page (Hills & Bagnall, UEA); the shapelet transform paper was not reachable in full text.
Full record
Benchmark task
Classify a length-512 outline series into one of 60 shape classes.
Who would use it
none applied; a standardisation benchmark for shape descriptors, re-used as a TSC benchmark
Task construction
passnone (source task label)
The benchmark label is the source's shape class, kept for 60 of the 70 MPEG-7 classes (which ten were dropped is undocumented); the source's class-scored retrieval counts as the source task (MPEG-7 policy); label origin: source task.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
failexact cross-split matches (2 of 600 test series)
Split sources
Archive default split, 10 train / 10 test per class as the archive page states S; who built it and whether source objects are kept apart is not documented RDR
SmallKitchenAppliancesDEVICE · Energy & appliances · 375/375 train/test · length 720 · 3 classesreal tasksource task labelUnclearassumed new case: new appliances or householdstemporaltimefamily: household-electricity-surveyopen question
Taskreal task
Source task
real task Household electricity monitoring for energy policy (Household Electricity Survey); each metered channel is annotated with its appliance, but the survey posed no classification task
Archive task
which small kitchen appliance produced the day's consumption trace (kettle, microwave, toaster)
Label attached by
Collectorssource taskSR
Verdict
real task
The survey's installing electricians annotated each monitored product with its appliance ("selected from a dropdown list"), so under policy L3 we count appliance-type recognition as the source's own task although the survey trained no classifier; the classification problem was posed on those annotations by the curators (Lines & Bagnall, SDM 2014; Lines 2015 thesis). Each input is an appliance trace metered on its own, so the task is appliance recognition, not whole-house disaggregation.
SplitUnclear
New case assumed
New appliances or households
Exact duplicates
none found D
Why
Same as Computers. [Computers: The source measured each physical appliance over many days. We do not know whether all days from an appliance or household were kept in one set. If the same appliance appears in both sets, the model may recognise its usual pattern. That would be a narrower test than recognising appliance types in new homes.]
Source
These series come from the UK government-sponsored household electricity monitoring study known as "Powering the Nation" (the DECC/Defra/Energy Saving Trust Household Electricity Survey).
Publication
UK Household Electricity Survey (DECC, Defra and the Energy Saving Trust, 2010-2011); Intertek Report R66141, "Household Electricity Survey: A study of domestic electrical product usage", final report issue 4. Household counts differ: the archive page, the thesis and Intertek (p. 24) give 251 households, Report 5 (p. 4) and gov.uk give 250; the 26 year-long households were sampled every 10 minutes except from late November 2010 to February 2011, when they recorded every 2 minutes (Intertek p. 64). Curated into TSC datasets by Lines & Bagnall (UEA): J. Lines, PhD thesis, UEA, 2015, Sec. 3.2.2 and Table 3.3 ("We provide five new classification problems that focus on domestic appliance classification from electricity consumption"); introduced in Lines & Bagnall, "Ensembles of elastic distance measures for time series classification", SIAM SDM 2014 (not read: no full text found).
Full record
Benchmark task
Classify a 720-point daily consumption profile as kettle, microwave or toaster.
Who would use it
UK energy policy (the survey); the curators' stated aim was to extract more from smart-meter data by "classifying appliances according to their electricity usage profiles". The derived task (naming a device from its own metered daily profile) is appliance recognition, not disaggregation of a whole-house load.
Task construction
passnone (source task label)
The benchmark label is the survey's appliance annotation of the metered channel (which small kitchen appliance produced the day's consumption trace (kettle, microwave, toaster)); label origin: source task, by the convention that a collectors' annotation of the quantity a deployed model would predict counts as the source task.
Data origin
temporalindex: time
Archive changes
windowingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split, made by the curators (Lines & Bagnall), who say these datasets were "split in a certain manner to avoid introducing bias" from one device appearing in both train and test S; the rule used is not stated and the files carry no household or device identifiers RIR
five appliance-group tasks cut from the UK Household Electricity Survey (ElectricDevices comes from a different study, the Visual Energy Trial); whether they share households or appliances is not documented
Overall verdict
real task; leakage unknown
New case assumed
New appliances or households.
Split evidence
each series is one day cut from a month (or, for some households, a year) of metering, so one appliance can supply many series R; which households, appliances and days entered the dataset is not documented. The curators say the split was made to avoid device bias, but give no rule, and the files carry no device identifiers, so device- or household-disjointness is neither documented nor refuted and sharing is not counted as leakage R. An appliance-recognition model meeting unseen appliances is the deployment case I. For SmallKitchenAppliances the survey metered 68 toasters against 250 toaster series here, so devices repeat R; whether across the split is not known.
Open questions
Appliance-type recognition is counted as the source task by convention (the survey annotated the channels but posed no classification task); a stricter reading files it as metadata. Appliance/household sharing inferred from the survey design but not documented, so Unclear rather than Concern.
SmoothSubspaceSIMULATED · Simulated · 150/150 train/test · length 15 · 3 classesfailstoy source taskGeneratedassumed new case: new generated examplessyntheticgenerator step
Taskfails
Source task
toy task Cluster identity, to test a k-means variant
Archive task
Which five-step window carries the signal
Label attached by
Generatorsource taskS
Verdict
failstoy source task
The source data were staged or generated to demonstrate a method.
SplitGenerated
New case assumed
New generated examples
Exact duplicates
none found D
Why
This is a generated task rather than measurements from people, devices or specimens. The corresponding source-separation question is therefore not assessed here. Its split would instead need to be examined through the generation procedure; synthetic origin alone is not evidence of leakage.
Source
No real-world problem exists. The data was generated by Huang et al. to test a clustering algorithm - specifically whether a k-means variant can recover "smooth subspaces", i.e. contiguous stretches of time stamps that carry the cluster signal while the remaining stamps are noise.
Publication
Huang, Ye, Xiong, Lau, Jiang, Wang, "Time series k-means - A new k-means type smooth subspace clustering for time series data", Information Sciences 367-368 (2016) 1-13. The primary paper was not reached (ScienceDirect paywall, article S0020025516303796); the generator description comes from the archive page, which was written by the data editor and cites that paper - "The data was originally intended for testing whether a clustering algorithm is able to extract smooth subspaces for clustering time series data [1]. There are 3 classes corresponding to which cluster the time series belong to. Each time series contain a continuous subspace spanning over 5 continuous time stamps. ... The rest of the time series are randomly generated. Data created by Xiaohui Huang et al. (see [1]). Data edited by Hoang Anh Dau."
Full record
Benchmark task
Classify a univariate length-15 series into the three generated cluster classes.
Who would use it
none (synthetic demo)
Task construction
failtoy source task
Benchmark target: Which five-step window carries the signal. Source target or purpose: Cluster identity, to test a k-means variant. Label attached by: Generator. Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question.
Data origin
syntheticindex: generator step
Archive changes
simulation
Leakage
n/a (generated)not assessed (generated data)
Split sources
Archive default split; who built it and by what rule is not documented RS
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails task construction (generated data)
New case assumed
New generated examples.
Split evidence
generated data: no deployment unit.
Label
generated data (clustering ground truth reused as classes). The generator's cluster identity (built to test a k-means variant) is reused as the class R.
real task walking surface under the robot (cement vs carpet)
Archive task
walking surface under the robot (cement vs carpet)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitConcern
New case assumed
New recording occasions
Exact duplicates
none found D
Why
The audit describes short pieces from one continuous robot recording per surface. Training and testing therefore share the robot run and its recording conditions. The test checks more pieces from those runs. It does not show whether surface recognition works after collecting another recording on a separate occasion.
Source
Vail & Veloso wanted a legged robot to recognize what it is walking on, so that it can switch to a gait parameter set tuned for that surface.
Publication
Vail & Veloso, "Learning from Accelerometer Data on a Legged Robot", 5th IFAC/EURON Symposium on Intelligent Autonomous Vehicles, 2004 (read in full). Sec. 1.1: "In addition to its camera, the robot also has an accelerometer. The accelerometer returns a real valued estimate of the robot's acceleration along the x, y, and z axes. The values are limited to the range of [-2,2] gravities for each axis and are sampled at 125 hertz. We define positive x as the direction that the robot is facing". Sec. 2: "We present experimental results showing the robots performance discriminating between a cement floor, the carpet in our laboratory, and the carpet on a RoboCup field." Sec. 2.2: "To test our surface prediction algorithm, we gathered five minute segments of accelerometer readings while the robot marched in place on each surface." Sec. 2.1: "Since the data arrives as a stream, we window it into discrete chunks when creating the feature vector for the classifier... We chose to use a one second sliding window."
Full record
Benchmark task
Classify a length-70 single-axis accelerometer window as recorded on cement or on carpet.
Who would use it
legged-robot developers doing online gait adaptation — Vail & Veloso frame it as choosing "a pregenerated set of parameters for movement on each particular surface"
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (walking surface under the robot (cement vs carpet)); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Archive default split; who built it and by what rule is not documented RSR
Multichannel source
yes, one channel from a multichannel recording
The source recorded three-axis accelerometer stream from a walking Sony AIBO ERS-210a quadruped robot; the UCR release keeps a single channel of it. Two of the three accelerometer axes are discarded, so the vertical impact signature — arguably the most informative channel for substrate hardness — is not in the data. Each instance is a sub-second window cut from a five-minute continuous recording, so gait-cycle phase within a window is arbitrary. Z-normalization removes the absolute acceleration scale. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
sub-second windows of one continuous five-minute recording per surface; the original authors used blocked cross-validation for this reason; the robot is deployed on a new day and floor patch R.
real task walking surface under the robot (cement vs carpet/field)
Archive task
walking surface under the robot (cement vs carpet/field)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitConcern
New case assumed
New recording occasions
Exact duplicates
none found D
Why
Same as SonyAIBORobotSurface1. [SonyAIBORobotSurface1: The audit describes short pieces from one continuous robot recording per surface. Training and testing therefore share the robot run and its recording conditions. The test checks more pieces from those runs. It does not show whether surface recognition works after collecting another recording on a separate occasion.]
Source
Vail & Veloso wanted a legged robot to recognize what it is walking on, so that it can switch to a gait parameter set tuned for that surface.
Publication
Vail & Veloso, "Learning from Accelerometer Data on a Legged Robot", 5th IFAC/EURON Symposium on Intelligent Autonomous Vehicles, 2004 (read in full). Sec. 1.1: "In addition to its camera, the robot also has an accelerometer. The accelerometer returns a real valued estimate of the robot's acceleration along the x, y, and z axes. The values are limited to the range of [-2,2] gravities for each axis and are sampled at 125 hertz. We define positive x as the direction that the robot is facing". Sec. 2: "We present experimental results showing the robots performance discriminating between a cement floor, the carpet in our laboratory, and the carpet on a RoboCup field." Sec. 2.2: "To test our surface prediction algorithm, we gathered five minute segments of accelerometer readings while the robot marched in place on each surface." Sec. 2.1: "Since the data arrives as a stream, we window it into discrete chunks when creating the feature vector for the classifier... We chose to use a one second sliding window."
Full record
Benchmark task
Classify a length-65 single-axis accelerometer window as recorded on cement or on carpet/field.
Who would use it
legged-robot developers doing online gait adaptation — Vail & Veloso frame it as choosing "a pregenerated set of parameters for movement on each particular surface"
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (walking surface under the robot (cement vs carpet/field)); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Archive default split; who built it and by what rule is not documented RSR
Multichannel source
yes, one channel from a multichannel recording
The source recorded three-axis accelerometer stream from a walking Sony AIBO ERS-210a quadruped robot; the UCR release keeps a single channel of it. Two of the three accelerometer axes are discarded, so the vertical impact signature — arguably the most informative channel for substrate hardness — is not in the data. Each instance is a sub-second window cut from a five-minute continuous recording, so gait-cycle phase within a window is arbitrary. Z-normalization removes the absolute acceleration scale. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
sub-second windows of one continuous five-minute recording per surface; the original authors used blocked cross-validation for this reason; the robot is deployed on a new day and floor patch R.
StarLightCurvesSENSOR · Earth & space science · 1,000/8,236 train/test · length 1024 · 3 classesreal tasksource task labelSeparatedassumed new case: new starsordered, non-temporalphase (folded light curve)open question
Taskreal task
Source task
real task variable-star class assigned by an expert (class names not documented in the linked sources)
Archive task
variable-star class assigned by an expert (three classes)
Label attached by
Collectorssource taskSR
Verdict
real task
Variable-star classification is a standing task of survey astronomy (catalogue construction); the expert labels came with the data, while the linked papers used it for indexing and anomaly detection.
SplitSeparated
New case assumed
New stars
Exact duplicates
none found D
Why
The source-level record describes one separate star per series. On that description, the test contains stars absent from training, rather than another piece of a familiar star's recording. No shared-star concern is identified. Possible shared observing conditions or processing procedures were not separately checked.
Source
Automated classification of variable stars. Sky surveys produce brightness measurements of hundreds of millions of celestial objects, far more than astronomers can inspect; the standing task is to assign each variable star to its physical type from the shape of its light curve.
Publication
Keogh, Wei, Xi, Lee & Vlachos, "LB_Keogh Supports Exact Indexing of Shapes under Rotation Invariance with Arbitrary Representations and Distance Measures", VLDB 2006 (expanded version, read in full at cs.ucr.edu/~eamonn/VLDB2006_Expanded.pdf), Sec. 2.4 "Indexing Star Light Curves": "A star light curve is a time series of brightness of a celestial object as a function of time." And, decisively for the x-axis: "There are two things which make this difficult. First is the enormous volume of data, the second is the fact that while is it possible to extract a single period of a light curve, there is no natural starting point. In order the find the similarity of two light curves it is therefore necessary to compare every possible circular shift of the data [29], which as we show below corresponds exactly to the rotation invariance matching problem for shapes in the one-dimensional representation." The archive page confirms the folding: "This is a dataset with 1,000 (purportedly) phase-aligned starlight curves of length 1,024". The archive's other source link, Rebbapragada, Protopapas, Brodley & Alcock, "Finding Anomalous Periodic Time Series" (arXiv:0905.3428), was reached only at abstract level; it likewise stresses that "the periods of any given pair of light-curves may be out of sync".
Full record
Benchmark task
Classify a length-1024 phase-folded stellar light curve into one of three expert-assigned variable-star classes.
Who would use it
survey astronomy — automated catalogue construction and anomaly triage for surveys producing more light curves than can be inspected by hand
Task construction
passnone (source task label)
The benchmark label is the expert-assigned variable-star class that came with the data from the Harvard Time Series Center (class names not documented in the linked sources); label origin: source task.
Index is phase: each series is a light curve folded on the star's period, so the ordered axis is phase within one cycle, not elapsed time.
Archive changes
resamplingz-normalization
Leakage
passsource separation by construction (each series is a distinct star)
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; split separated by construction
New case assumed
New stars.
Split evidence
each folded curve is one star; deployment classifies new stars, which a split over objects provides R. The index is phase of a folded light curve (period estimation done upstream and not released); filed as ordered, non-temporal R.
Open questions
Index is phase; a reader who regards a phase-folded curve as a time axis would count it as temporal.
real task authentic strawberry puree vs non-strawberry (adulterated purees and other fruits)
Archive task
authentic strawberry puree vs non-strawberry (adulterated purees and other fruits)
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitConcern
New case assumed
New physical samples
Exact duplicates
1 of 370 test series (0.3%) exactly match a training series D; 1 duplicate groups have conflicting labels
Why
The test set is the source study's own independent test set (370 of the 983 spectra held out). However, the value check finds one sequence present in both sets with different labels. Thus, the original split description does not resolve this inconsistency in the released data.
Source
An industrial food-fraud screening problem at the Institute of Food Research, Norwich: strawberry puree is expensive and is diluted with cheaper fruit or sugar, so a rapid instrumental test for "is this really strawberry?" was wanted. Abstract (J.
Publication
Holland, Kemsley, Wilson, "Use of Fourier transform infrared spectroscopy and partial least squares regression for the detection of adulteration of strawberry purees", J. Sci. Food Agric. 76, 263-269, 1998, doi:10.1002/(SICI)1097-0010(199802)76:2<263::AID-JSFA943>3.0.CO;2-F. Abstract read via the Europe PMC REST API; full text not reached. The donor's own page, IFR Analytical Sciences "Example Datasets for Download", describes the release: "983 Mid-infrared (MIR) spectra of fresh fruit purees: strawberry (authentic samples) and non-strawberry (adulterated strawberries and other fruits). Raw data matrix size [983 x 235]."
Full record
Benchmark task
Classify a univariate length-235 mid-IR spectrum as strawberry or non-strawberry puree.
Who would use it
fruit-processing industry QA and food-fraud enforcement labs
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (authentic strawberry puree vs non-strawberry (adulterated purees and other fruits)); label origin: source task.
Data origin
ordered, non-temporalindex: wavelength or wavenumber
Archive changes
spectrum-as-series
Leakage
failexact cross-split matches (1 of 370 test series, 1 duplicate groups with conflicting labels)
Split sources
The 370-series test set is the source paper's own independent test set (Holland et al. 1998) RDR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails leakage
New case assumed
New physical samples.
Split evidence
the 370-case test set is the source paper's own independent test set R. The single cross-split match carries conflicting labels, so the concern is label inconsistency in the released files, not memorisation; the source study's own independent test set is otherwise documented D.
Open questions
The single cross-split match has conflicting labels (label inconsistency rather than memorisation).
The source study posed a classification target that a stakeholder outside the paper needed.
SplitConcern
New case assumed
New physical specimens
Exact duplicates
2 of 625 test series (0.3%) exactly match a training series D
Why
The cases are outlines derived from leaf images. The description does not establish how physical specimens and any repeated views were assigned to the two sets. New-specimen evaluation therefore cannot be verified; repeated views are a possibility to check, not an observed overlap. Separately, the value check reports two test cases identical to training data.
Source
A computer-vision project at Linköping University, proposed by the Swedish Museum of Natural History: identify Swedish trees from scanned images of single leaves. Data collected for Oskar Söderkvist's MSc thesis "Computer vision classification of leaves from Swedish trees" (2001), covering 15 tree classes (Ulmus carpinifolia, Acer, Fagus silvatica, ...).
Publication
Söderkvist, O. "Computer vision classification of leaves from Swedish trees", MSc thesis, Linköping University, 2001
Full record
Benchmark task
Classify a univariate length-128 leaf-outline series into one of 15 tree classes.
Who would use it
botanists / automated plant identification
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (tree class (15 classes)); label origin: source task. Which archive class number is which thesis class is undocumented.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingz-normalization
Leakage
failexact cross-split matches (2 of 625 test series)
Split sources
Archive default split; who built it and by what rule is not documented RDR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails leakage
New case assumed
New physical specimens.
Split evidence
one outline per leaf; split composition undocumented R.
toy task Purpose undocumented; a copying experiment with 13 participants donated by the archive's curators, without a publication
Archive task
Six classes; three symbols documented
Label attached by
Collectors (= curators)source taskR
Verdict
failstoy source task
The data were donated by J. Brady and E. Keogh (one of the archive's curators) from a drawing experiment whose purpose no source records; we file it as a toy source because no application or external stakeholder is documented.
SplitUnclear
New case assumed
New people
Exact duplicates
none found D
Why
The source contains several symbol-drawing attempts per person. We do not know whether all attempts from a person were kept in one set. A test attempt could therefore come from a person already represented in training. This possible sharing needs participant identifiers to check; performance on new people is not established.
Source
A small drawing experiment, described only by the archive page: "Thirteen people participated in this experiment. They were asked to copy the randomly appearing symbol as best they could.
Publication
Archive page only (donated by J. Brady and E. Keogh; no data_source_link or second_link, and no publication located). Provenance is therefore unsupported beyond the archive's own text.
Full record
Benchmark task
Classify a length-398 x-motion series into one of six classes.
Who would use it
none identified; on the face of it this is a copying/drawing experiment created to produce a TSC benchmark
Task construction
failtoy source task
Benchmark target: Six classes; three symbols documented. Source target or purpose: Purpose undocumented; a copying experiment with 13 participants donated by the archive's curators, without a publication. Label attached by: Collectors (= curators). Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question.
Data origin
temporalindex: time
Archive changes
channel-selectionz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded the horizontal component of hand motion recorded while a person copies a symbol; the UCR release keeps a single channel of it. Only the x-component of a 2D drawing motion is kept; the y-component, pressure and any pen-up/pen-down structure are gone, so the same symbol drawn with a different orientation or stroke order gives a different series. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
Dataset family
none from the same source task
Overall verdict
fails task construction (leakage unknown)
New case assumed
New people.
Split evidence
13 subjects with about 30 attempts each; no subject-disjoint split documented R.
Label
class = symbol/drawing condition; documented only by the archive page R. Documented only by the archive page (13 participants copying symbols) R.
Open questions
Documented by the archive page only; the purpose of the experiment is unrecorded, so 'toy source' is our reading.
SyntheticControlSIMULATED · Simulated · 300/300 train/test · length 60 · 6 classesfailstoy source taskGeneratedassumed new case: new generated examplessyntheticgenerator step
Taskfails
Source task
toy task None; no process measured, generated to test a similarity method
Archive task
Six control-chart patterns
Label attached by
Generatorsource taskS
Verdict
failstoy source task
The source data were staged or generated to demonstrate a method.
SplitGenerated
New case assumed
New generated examples
Exact duplicates
none found D
Why
This is a generated task rather than measurements from people, devices or specimens. The corresponding source-separation question is therefore not assessed here. Its split would instead need to be examined through the generation procedure; synthetic origin alone is not evidence of leakage.
Source
No real measurements exist. The six classes imitate the pattern vocabulary of statistical process control - the shapes a quality engineer is trained to spot on a manufacturing control chart (in control, cyclic, trending up/down, shifted up/down).
Publication
UCI KDD Archive dataset page (read), which names the generator - "Original Owner and Donor Dr Robert Alcock ... Date Donated - June 8, 1999. Data Characteristics - This dataset contains 600 examples of control charts synthetically generated by the process in Alcock and Manolopoulos (1999). ... The classes are organized as follows - 1-100 Normal, 101-200 Cyclic, 201-300 Increasing trend, 301-400 Decreasing trend, 401-500 Upward shift, 501-600 Downward shift". The primary conference paper (Alcock and Manolopoulos, 7th Hellenic Conference on Informatics, Ioannina 1999) was not reached, but its generating equations are reproduced in Geurts (2002) PhD thesis, App. B.2.4 "CC" (read - taking m = 30, s = 2 and r(t) uniform on [-3,3], Normal is m + s*r(t), Cyclic adds a*sin(2*pi*t/T) with a and T drawn in [10,15], Increasing adds g*t with g in [0.2,0.5], Upward adds a step x in [7.5,20] at a random time in [n/3, 2n/3], and the down-variants negate these).
Full record
Benchmark task
Classify a univariate length-60 series into the six control-chart pattern classes.
Who would use it
none (synthetic demo). The class vocabulary is borrowed from control-chart pattern recognition in manufacturing quality control - whose real users would be process/quality engineers - but no process was ever measured; the UCI page cites Pham and Chan (1998) on control-chart pattern recognition as background.
Task construction
failtoy source task
Benchmark target: Six control-chart patterns. Source target or purpose: None; no process measured, generated to test a similarity method. Label attached by: Generator. Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question.
Data origin
syntheticindex: generator step
Archive changes
simulation
Leakage
n/a (generated)not assessed (generated data)
Split sources
Archive default split; who built it and by what rule is not documented RS
The source collected the data but posed no classification target.
SplitConcern
New case assumed
New people or recording occasions
Exact duplicates
4 of 228 test series (1.8%) exactly match a training series D
Why
The source contains repeated motion recordings from a small group of actors. We suspect that actors are shared between the sets, but the files lack actor identifiers for checking this. We therefore cannot establish performance on new people or new recording occasions, and should not present actor overlap as verified. Separately, the value check finds four test cases identical to training data.
Source
There is no original prediction problem. The series come from the CMU Graphics Lab Motion Capture Database, a general-purpose optical motion-capture resource built for graphics and animation research, in which each recorded take carries a free-text motion description.
Publication
CMU Graphics Lab Motion Capture Database (http://mocap.cs.cmu.edu/) as the data source, and Ye & Keogh, "Time series shapelets: a novel technique that allows accurate, interpretable and fast classification" (Data Mining and Knowledge Discovery, 2011) as the first use, per the archive page. The KDD 2009 conference version of the shapelet paper (cs.ucr.edu/~eamonn/shaplet.pdf) was read and does not describe this dataset; the DMKD 2011 journal version, which does, is paywalled and was not reached.
Full record
Benchmark task
Classify a length-277 single-axis (X axis) marker trajectory from a walking motion as normal or abnormal. 40 train / 228 test.
Who would use it
Nominally gait analysis / clinical or surveillance gait screening; in practice this was assembled as a shapelet-interpretability demonstration, and the actors are not impaired.
Task construction
failtask replacement
Benchmark target: Normal vs acted abnormal gait. Source target or purpose: CMU motion descriptions; no defined task. Label attached by: Curators. Mechanism 'task replacement': the archive replaces the source's label with a different task, or defines classes by its own rule where the source posed none.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
failexact cross-split matches (4 of 228 test series)
Split sources
Archive default split; who built it and by what rule is not documented RDIR
Multichannel source
yes, one channel from a multichannel recording
The source recorded optical motion capture of an actor walking; the UCR release keeps a single channel of it. Motion capture gives a full 3D skeleton over time; this dataset keeps one coordinate of one marker, so the coordination between limbs - which is what actually distinguishes a limp from a normal gait - is gone. Truncation to a common length crops the ends of longer segments instead of resampling them, and z-normalisation removes stride amplitude. The source recording is multivariate, though the source posed no classification problem; no multivariate version of it is in UEA.
the X and Y axes of the same body of CMU walking motions (case correspondence undocumented)
Overall verdict
fails task construction and leakage
New case assumed
New people or recording occasions.
Split evidence
a small number of CMU actors with many takes each R; no subject id is released and the archive does not state whether takes of one actor were kept in one set, so actor sharing is inferred rather than observed; a gait screen is deployed on new people I.
Label
labels derived by the curators (Ye & Keogh) by keyword matching against CMU's free-text motion descriptions, as the archive page states S. The Concern verdict rests on the four exact cross-split matches D; actor sharing is inferred from the source design (few CMU actors, many takes) and not observed I.
Open questions
Concern rests on the exact cross-split matches alone; actor sharing is not established (ToeSegmentation2 has no match and is Unclear).
ToeSegmentation2MOTION · Human motion & gesture · 36/130 train/test · length 343 · 2 classesfailstask replacementUnclearassumed new case: new people or recording sessionstemporaltimesingle channelfamily: toe-segmentationopen question
Taskfails
Source task
none CMU motion descriptions; no defined task
Archive task
Normal vs acted abnormal gait
Label attached by
CuratorsredefinedS
Verdict
failstask replacement
The source collected the data but posed no classification target.
SplitUnclear
New case assumed
New people or recording sessions
Exact duplicates
none found D
Why
Same uncertainty about the actors and recording occasions as ToeSegmentation1. The four value matches mentioned there concern ToeSegmentation1, not this dataset. [ToeSegmentation1: The source contains repeated motion recordings from a small group of actors. We suspect that actors are shared between the sets, but the files lack actor identifiers for checking this. We therefore cannot establish performance on new people or new recording occasions, and should not present actor overlap as verified. Separately, the value check finds four test cases identical to training data.]
Source
There is no original prediction problem. The series come from the CMU Graphics Lab Motion Capture Database, a general-purpose optical motion-capture resource built for graphics and animation research, in which each recorded take carries a free-text motion description.
Publication
CMU Graphics Lab Motion Capture Database (http://mocap.cs.cmu.edu/) as the data source, and Ye & Keogh, "Time series shapelets: a novel technique that allows accurate, interpretable and fast classification" (Data Mining and Knowledge Discovery, 2011) as the first use, per the archive page. The KDD 2009 conference version of the shapelet paper (cs.ucr.edu/~eamonn/shaplet.pdf) was read and does not describe this dataset; the DMKD 2011 journal version, which does, is paywalled and was not reached.
Full record
Benchmark task
Classify a length-343 single-axis (Y axis) marker trajectory from a walking motion as normal or abnormal. 36 train / 130 test.
Who would use it
Nominally gait analysis / clinical or surveillance gait screening; in practice this was assembled as a shapelet-interpretability demonstration, and the actors are not impaired.
Task construction
failtask replacement
Benchmark target: Normal vs acted abnormal gait. Source target or purpose: CMU motion descriptions; no defined task. Label attached by: Curators. Mechanism 'task replacement': the archive replaces the source's label with a different task, or defines classes by its own rule where the source posed none.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RIR
Multichannel source
yes, one channel from a multichannel recording
The source recorded optical motion capture of an actor walking; the UCR release keeps a single channel of it. Motion capture gives a full 3D skeleton over time; this dataset keeps one coordinate of one marker, so the coordination between limbs - which is what actually distinguishes a limp from a normal gait - is gone. Truncation to a common length crops the ends of longer segments instead of resampling them, and z-normalisation removes stride amplitude. The source recording is multivariate, though the source posed no classification problem; no multivariate version of it is in UEA.
the X and Y axes of the same body of CMU walking motions (case correspondence undocumented)
Overall verdict
fails task construction (leakage unknown)
New case assumed
New people or recording sessions.
Split evidence
a small number of CMU actors with many takes each R; no subject id is released and the archive does not state whether takes of one actor were kept in one set, so actor sharing is inferred rather than observed; a gait screen is deployed on new people I.
Label
as ToeSegmentation1. No exact match; actor sharing inferred but not observed, hence Unclear I. The archive page states that the 'abnormal' gaits are acted and that the labels were derived from CMU's free-text motion descriptions S.
Open questions
Actor sharing inferred but not observed, so Unclear rather than Concern.
toy task Sixteen simulated plant transient/failure classes
Archive task
simulated instrumentation-failure class (4-class subset of 16)
Label attached by
Generatorsource taskS
Verdict
failstoy source task
The source data were staged or generated to demonstrate a method.
SplitSeparated
New case assumed
New simulated events
Exact duplicates
none found D
Why
Each series is described as a separate simulated event, so testing uses other generated events rather than another piece of one recording. No shared-event issue is identified by that description. Whether the generation process creates further dependencies would need its settings or code to check.
Source
Roverso's Transient Classification Benchmark (OECD Halden Reactor Project): simulated instrumentation-failure transients of a nuclear power plant, 16 fault classes with 50 four-feature instances each, built to compare transient classifiers; the archive uses a four-class, one-feature subset.
Publication
timeseriesclassification.com description page, which is the donor's own account: "This 4-class dataset is a subset of the Transient Classification Benchmark (trace project), an initiative at the turn of the century to collate data from the application domain of the process industry (e.g. nuclear, chemical, etc.). It is a synthetic dataset designed to simulate instrumentation failures in a nuclear power plant, created by Davide Roverso. The full dataset consists of 16 classes, 50 instances in each class. Each instance has 4 features. The TRACE subset only uses the second feature of class 2 and the third feature of class 3 and 7. Hence, this dataset contains 200 instances, 50 for each class. All instances are linearly interpolated to have the same length of 275 data points, and are z-normalized." Roverso's primary publication was not read: the archive's CiteSeerX link (doi 10.1.1.33.7261) now resolves only to a Wayback error page, and the corresponding journal article (Roverso, "Soft computing tools for transient classification", Information Sciences 127(3-4), 2000) is behind Elsevier's paywall (HTTP 403). The synthetic origin therefore rests on the donor-written archive page alone.
Full record
Benchmark task
Classify a length-275 simulated instrument trace into one of four curator-selected classes.
Who would use it
none directly (simulated benchmark data); the motivating stakeholder is nuclear/process-plant operations, where transient identification supports operator decision aids
Task construction
failtoy source task
Benchmark target: simulated instrumentation-failure class (4-class subset of 16). Source target or purpose: Sixteen simulated plant transient/failure classes. Label attached by: Generator. Mechanism 'toy source task': the data were simulated by method developers and the classes chosen to demonstrate a method, not to answer an external question. The archive keeps four classes, one feature each; that this redefines the classes as feature/fault pairs is inferred, not documented.
Data origin
temporalindex: time
Simulated process (nuclear-plant simulator) indexed by simulated time; a toy source task whose series index is nevertheless time.
passsource separation by construction (each series is a distinct simulated transient)
Split sources
Donor's (D. Roverso) four-class subset, 100 train / 100 test, one simulated transient per series; rule undocumented RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded simulated instrument traces from a nuclear-power-plant process simulator, four measured features per transient; the UCR release keeps a single channel of it. The values come from a plant simulator, so the noise model, fault dynamics and sensor characteristics are all modelling choices. The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
Dataset family
none from the same source task
Overall verdict
fails task construction
New case assumed
New simulated events.
Split evidence
each series is one simulated transient (50 per class); the split is over transients R.
Label
the four-class subset was donated by the benchmark's own author (D. Roverso) and the archive text, quoted verbatim, is the donor's account S; that the four classes are feature/fault pairs is inferred R. Simulator output of a nuclear-plant process model; treated as a toy source (classes chosen to exercise methods) but as a temporal signal with one simulated transient per series R; the primary source is unreachable.
Open questions
The archive text names three feature/class pairs for four classes, so the class mapping is unclear; label-by set to Generator (a subset of Roverso's fault classes) pending that mapping. Filed as a toy source but its split is assessed as Separated (one simulated transient per series), not Generated. Toy task kept on the donor's account and the abstract of Roverso (2000), which frames the data as a comparison of neural network designs motivated by plant alarm handling; Roverso's full papers were not read (paywalled or refused), and which of his experiments the Trace data come from is unverified.
The source collected the data but posed no classification target.
SplitConcern
New case assumed
Independent recordings
Exact duplicates
none found D
Why
The label identifies which ECG lead produced the signal. Training and test beats come from one patient's recording. The model may recognise differences specific to those leads during that measurement. The test does not show that the same distinction works when another patient or recording is used.
Source
The source is the MIT-BIH Long-Term ECG Database on PhysioNet, which the PhysioNet page describes as "7 long-duration electrocardiogram (ECG) recordings (14 to 22 hours each), with manually reviewed beat annotations" - a resource for long-term arrhythmia analysis.
Publication
timeseriesclassification.com description page and PhysioNet MIT-BIH Long-Term ECG Database (https://physionet.org/content/ltdb/1.0.0/ and the 15814 header), both read 2026-08. No paper posing this classification problem was found.
Full record
Benchmark task
Given a length-82 heartbeat from a single ten-minute ECG of one patient, decide which of the two leads it came from.
Who would use it
The LTDB itself supports arrhythmia-detection research; the archive's lead-identity task has no clinical user.
Task construction
failmetadata as label
Benchmark target: Which of two ECG leads. Source target or purpose: None used; one long-term ECG record. Label attached by: Curators. Mechanism 'metadata as label': the archive derives the classes from a recording attribute.
Data origin
temporalindex: time
Archive changes
channel-selectionwindowingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Archive default split; who built it and by what rule is not documented RS
Multichannel source
yes, one channel from a multichannel recording
The source recorded multi-lead long-term ambulatory ECG of one 58-year-old male; the UCR release keeps a single channel of it. Each beat is presented as an isolated univariate series, so the pairing between the two leads of the same beat - the information a cardiologist actually uses when reading a multi-lead ECG - is thrown away. Z-normalisation removes absolute amplitude, which is one of the main things that differs between leads; what remains is lead-specific waveform morphology (R/S ratio, T-wave polarity). The problem the source posed is multivariate, and its place is the multivariate archive; no multivariate version of it is in UEA.
Dataset family
none from the same source task
Overall verdict
fails task construction and leakage
New case assumed
Independent recordings.
Split evidence
one patient, one ten-minute window; every training and test beat comes from the same recording, minutes apart S.
Label
lead label assigned by the curator (E. Keogh) over a PhysioNet recording S. The class is the ECG lead (signal 0 vs 1) of one ten-minute window of one PhysioNet long-term recording (ltdb 15814) S; no paper posing this task was found R.
TwoPatternsSIMULATED · Simulated · 1,000/4,000 train/test · length 128 · 4 classesfailstoy source taskGeneratedassumed new case: new generated examplessyntheticgenerator step
Taskfails
Source task
toy task None; designed for thesis experiments
Archive task
Ordered pair of up/down steps in noise
Label attached by
Generatorsource taskS
Verdict
failstoy source task
The source data were staged or generated to demonstrate a method.
SplitGenerated
New case assumed
New generated examples
Exact duplicates
none found D
Why
This is a generated task rather than measurements from people, devices or specimens. The corresponding source-separation question is therefore not assessed here. Its split would instead need to be examined through the generation procedure; synthetic origin alone is not evidence of leakage.
Source
No real-world problem exists. Pierre Geurts designed the "Two-pat" problem for the time-series part of his PhD thesis, to test whether a classifier can detect two local patterns and, crucially, their order.
Publication
Geurts, "Contributions to decision tree induction - bias/variance tradeoff and time series classification", PhD thesis, University of Liege, 2002, App. B.2.3 "Two-pat" (read, p. 237) - "This dataset has also be designed by us for our experiments. In this problem, each scenario is described by a single temporal input attribute A which is defined on the integer time axis 0,1,...,127"; the patterns are "s1 and s2 defining the patterns of respective lengths l1 and l2", "e(t) is some noise surrounding the patterns, drawn from a standard normal distribution N(0,1)", "l1 and l2 are integers uniformly drawn in [16;32]" and t1, t2 are drawn in [0;127] so that the two patterns do not overlap; the up/down steps take values -5 then +5 (us) or +5 then -5 (ds), each chosen with probability 0.5, and Eq. B.4 defines the four classes uu, ud, du, dd, which "are thus equiprobable".
Full record
Benchmark task
Classify a univariate length-128 series into the four ordered step-pair classes.
Who would use it
none (synthetic demo)
Task construction
failtoy source task
Benchmark target: Ordered pair of up/down steps in noise. Source target or purpose: None; designed for thesis experiments. Label attached by: Generator. Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question.
Data origin
syntheticindex: generator step
Archive changes
simulation
Leakage
n/a (generated)not assessed (generated data)
Split sources
Archive default split; who built it and by what rule is not documented RS
toy task None; unpublished generator (Grenoble LIG/AMA)
Archive task
Bell pointing up, absent, or pointing down
Label attached by
Generatorsource taskS
Verdict
failstoy source task
The source data were staged or generated to demonstrate a method.
SplitGenerated
New case assumed
New generated examples
Exact duplicates
none found D
Why
This is a generated task rather than measurements from people, devices or specimens. The corresponding source-separation question is therefore not assessed here. Its split would instead need to be examined through the generation procedure; synthetic origin alone is not evidence of leakage.
Source
No real-world problem exists. UMD is a synthetic three-class benchmark from the same donor as BME: a small bell appears near the start or the end of the series and the class is whether that bell points up (Up), is absent (Middle), or points down (Down).
Publication
No publication found. The only provenance is the donor line on the archive page: "Data source: University Joseph Fourier (Grenoble I), Laboratoire d'informatique de Grenoble(LIG), Equipe AMA" (timeseriesclassification.com, UMD), with no data_source_link. As for BME, the likely origin is the Grenoble AMA group (candidate: Frambourg, Douzal-Chouakria & Gaussier, IDA 2013), but that text could not be reached (HAL access denied, ResearchGate 403) and no consulted source states UMD was introduced there.
Full record
Benchmark task
Classify a univariate length-150 series as Up, Middle or Down.
Who would use it
none (synthetic demo)
Task construction
failtoy source task
Benchmark target: Bell pointing up, absent, or pointing down. Source target or purpose: None; unpublished generator (Grenoble LIG/AMA). Label attached by: Generator. Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question.
Data origin
syntheticindex: generator step
Archive changes
simulation
Leakage
n/a (generated)not assessed (generated data)
Split sources
Archive default split; who built it and by what rule is not documented RS
UWaveGestureLibraryAllHAR · Human motion & gesture · 896/3,582 train/test · length 945 · 8 classesreal tasksource task labelUnclearassumed new case: new days or recording occasionsordered, non-temporaltime (three axes concatenated)family: uwave
Taskreal task
Source task
real task which of the eight predefined vocabulary gestures was performed
Archive task
which of the eight predefined vocabulary gestures was performed
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New days or recording occasions
Exact duplicates
none found D
Why
The original system recognises gestures for familiar users, so sharing users can be appropriate. The missing information is whether all recordings from a day were kept together. Without this, the test could contain gestures recorded on a day already represented in training, rather than gestures collected when the user returns.
Source
Liu, Zhong, Wickramasuriya and Vasudevan built uWave, a personalized gesture-recognition system driven by a single three-axis accelerometer, so that consumer devices could be controlled by hand gestures with only one training sample per gesture.
Publication
Liu, Zhong, Wickramasuriya, Vasudevan, "uWave: Accelerometer-based Personalized Gesture Recognition and Its Applications", PerCom 2009 / Pervasive and Mobile Computing 5(6), 2009. Read in full (author copy, yecl.org/publications/liu09percom.pdf).
Full record
Benchmark task
Classify a length-945 series - the x, y and z accelerometer channels concatenated - as one of the eight uWave vocabulary gestures.
Who would use it
designers of gesture-based interaction for consumer electronics and mobile devices; also proposed by the authors for gesture-based user authentication
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of the eight predefined vocabulary gestures was performed); label origin: source task.
Data origin
ordered, non-temporalindex: time (three axes concatenated)
Three accelerometer axes concatenated end to end; within each block the index is time, across blocks it is not, so the series as a whole is ordered but not temporal.
Archive changes
concatenationz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive split 896 train / 3582 test over the whole uWave library; not documented as user- or day-disjoint; no ids ship RR
X, Y and Z are the three accelerometer axes of the same gestures, All concatenates them; UEA UWaveGestureLibrary is a multivariate subsample of the same library
Overall verdict
real task; leakage unknown
New case assumed
New days or recording occasions.
Split evidence
uWave is personalised (one training sample per gesture per user), so the right split is by day within user; the archive's 896/3582 split over the whole library is not documented as day-disjoint and no ids ship R. The x, y and z accelerometer channels are concatenated end to end; within each block the index is time R.
UWaveGestureLibraryXHAR · Human motion & gesture · 896/3,582 train/test · length 315 · 8 classesreal tasksource task labelUnclearassumed new case: new sessions or daystemporaltimesingle channelfamily: uwave
Taskreal task
Source task
real task which of the eight predefined vocabulary gestures was performed
Archive task
which of the eight predefined vocabulary gestures was performed
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New sessions or days
Exact duplicates
none found D
Why
Same as UWaveGestureLibraryAll. [UWaveGestureLibraryAll: The original system recognises gestures for familiar users, so sharing users can be appropriate. The missing information is whether all recordings from a day were kept together. Without this, the test could contain gestures recorded on a day already represented in training, rather than gestures collected when the user returns.]
Source
Liu, Zhong, Wickramasuriya and Vasudevan built uWave, a personalized gesture-recognition system driven by a single three-axis accelerometer, so that consumer devices could be controlled by hand gestures with only one training sample per gesture.
Publication
Liu, Zhong, Wickramasuriya, Vasudevan, "uWave: Accelerometer-based Personalized Gesture Recognition and Its Applications", PerCom 2009 / Pervasive and Mobile Computing 5(6), 2009. Read in full (author copy, yecl.org/publications/liu09percom.pdf).
Full record
Benchmark task
Classify a length-315 single-axis accelerometer series as one of the eight uWave vocabulary gestures.
Who would use it
designers of gesture-based interaction for consumer electronics and mobile devices; also proposed by the authors for gesture-based user authentication
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of the eight predefined vocabulary gestures was performed); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selectionz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive split 896 train / 3582 test over the whole uWave library; not documented as user- or day-disjoint; no ids ship RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded three-axis acceleration recorded by a hand-held Nintendo Wii remote; the gesture is delimited by the user pressing and releasing the 'A' button; the UCR release keeps a single channel of it. Two of the three accelerometer axes are discarded, and gestures such as "circle" or "square" are only weakly distinguishable from one axis. Gesture duration is normalized away by the fixed length 315; participant identity and recording day, both recorded in the original study, are not released. The problem the source posed is multivariate, and its place is the multivariate archive, where it already exists as UWaveGestureLibrary.
X, Y and Z are the three accelerometer axes of the same gestures, All concatenates them; UEA UWaveGestureLibrary is a multivariate subsample of the same library
Overall verdict
real task; leakage unknown
New case assumed
New sessions or days.
Split evidence
uWave is personalised (one training sample per gesture per user), so the right split is by day within user; the archive's 896/3582 split over the whole library is not documented as day-disjoint and no ids ship R.
UWaveGestureLibraryYHAR · Human motion & gesture · 896/3,582 train/test · length 315 · 8 classesreal tasksource task labelUnclearassumed new case: new sessions or daystemporaltimesingle channelfamily: uwave
Taskreal task
Source task
real task which of the eight predefined vocabulary gestures was performed
Archive task
which of the eight predefined vocabulary gestures was performed
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New sessions or days
Exact duplicates
none found D
Why
Same as UWaveGestureLibraryAll. [UWaveGestureLibraryAll: The original system recognises gestures for familiar users, so sharing users can be appropriate. The missing information is whether all recordings from a day were kept together. Without this, the test could contain gestures recorded on a day already represented in training, rather than gestures collected when the user returns.]
Source
Liu, Zhong, Wickramasuriya and Vasudevan built uWave, a personalized gesture-recognition system driven by a single three-axis accelerometer, so that consumer devices could be controlled by hand gestures with only one training sample per gesture.
Publication
Liu, Zhong, Wickramasuriya, Vasudevan, "uWave: Accelerometer-based Personalized Gesture Recognition and Its Applications", PerCom 2009 / Pervasive and Mobile Computing 5(6), 2009. Read in full (author copy, yecl.org/publications/liu09percom.pdf).
Full record
Benchmark task
Classify a length-315 single-axis accelerometer series as one of the eight uWave vocabulary gestures.
Who would use it
designers of gesture-based interaction for consumer electronics and mobile devices; also proposed by the authors for gesture-based user authentication
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of the eight predefined vocabulary gestures was performed); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selectionz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive split 896 train / 3582 test over the whole uWave library; not documented as user- or day-disjoint; no ids ship RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded three-axis acceleration recorded by a hand-held Nintendo Wii remote; the gesture is delimited by the user pressing and releasing the 'A' button; the UCR release keeps a single channel of it. Two of the three accelerometer axes are discarded, and gestures such as "circle" or "square" are only weakly distinguishable from one axis. Gesture duration is normalized away by the fixed length 315; participant identity and recording day, both recorded in the original study, are not released. The problem the source posed is multivariate, and its place is the multivariate archive, where it already exists as UWaveGestureLibrary.
X, Y and Z are the three accelerometer axes of the same gestures, All concatenates them; UEA UWaveGestureLibrary is a multivariate subsample of the same library
Overall verdict
real task; leakage unknown
New case assumed
New sessions or days.
Split evidence
uWave is personalised (one training sample per gesture per user), so the right split is by day within user; the archive's 896/3582 split over the whole library is not documented as day-disjoint and no ids ship R.
UWaveGestureLibraryZHAR · Human motion & gesture · 896/3,582 train/test · length 315 · 8 classesreal tasksource task labelUnclearassumed new case: new sessions or daystemporaltimesingle channelfamily: uwave
Taskreal task
Source task
real task which of the eight predefined vocabulary gestures was performed
Archive task
which of the eight predefined vocabulary gestures was performed
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitUnclear
New case assumed
New sessions or days
Exact duplicates
none found D
Why
Same as UWaveGestureLibraryAll. [UWaveGestureLibraryAll: The original system recognises gestures for familiar users, so sharing users can be appropriate. The missing information is whether all recordings from a day were kept together. Without this, the test could contain gestures recorded on a day already represented in training, rather than gestures collected when the user returns.]
Source
Liu, Zhong, Wickramasuriya and Vasudevan built uWave, a personalized gesture-recognition system driven by a single three-axis accelerometer, so that consumer devices could be controlled by hand gestures with only one training sample per gesture.
Publication
Liu, Zhong, Wickramasuriya, Vasudevan, "uWave: Accelerometer-based Personalized Gesture Recognition and Its Applications", PerCom 2009 / Pervasive and Mobile Computing 5(6), 2009. Read in full (author copy, yecl.org/publications/liu09percom.pdf).
Full record
Benchmark task
Classify a length-315 single-axis accelerometer series as one of the eight uWave vocabulary gestures.
Who would use it
designers of gesture-based interaction for consumer electronics and mobile devices; also proposed by the authors for gesture-based user authentication
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (which of the eight predefined vocabulary gestures was performed); label origin: source task.
Data origin
temporalindex: time
Archive changes
channel-selectionz-normalization
Leakage
unknownsource separation undocumented
Split sources
Archive split 896 train / 3582 test over the whole uWave library; not documented as user- or day-disjoint; no ids ship RR
Multichannel source
yes, one channel from a multichannel recording
The source recorded three-axis acceleration recorded by a hand-held Nintendo Wii remote; the gesture is delimited by the user pressing and releasing the 'A' button; the UCR release keeps a single channel of it. Two of the three accelerometer axes are discarded, and gestures such as "circle" or "square" are only weakly distinguishable from one axis. Gesture duration is normalized away by the fixed length 315; participant identity and recording day, both recorded in the original study, are not released. The problem the source posed is multivariate, and its place is the multivariate archive, where it already exists as UWaveGestureLibrary.
X, Y and Z are the three accelerometer axes of the same gestures, All concatenates them; UEA UWaveGestureLibrary is a multivariate subsample of the same library
Overall verdict
real task; leakage unknown
New case assumed
New sessions or days.
Split evidence
uWave is personalised (one training sample per gesture per user), so the right split is by day within user; the archive's 896/3582 split over the whole library is not documented as day-disjoint and no ids ship R.
real task whether the process run recorded by the sensor was normal or abnormal, as judged by a domain expert
Archive task
whether the process run recorded by the sensor was normal or abnormal, as judged by a domain expert
Label attached by
Source studysource taskS
Verdict
real task
The source study posed a classification target that a stakeholder outside the paper needed.
SplitConcern
New case assumed
New wafers or production runs
Exact duplicates
none found D
Why
The archive ships all six sensor traces of each wafer as separate cases and split them without regard to wafer or run. Matching every shipped series to the raw files shows 690 wafers and 99 of 104 production runs on both sides; every training series has another sensor of its wafer in test. The test therefore does not measure performance on new wafers or runs.
Source
Semiconductor manufacturing process control. Plasma-etch tools used to fabricate silicon wafers carry large numbers of in-line sensors; if a tool drifts outside tolerance during any of the 250-plus processing steps, yield and reliability suffer or the wafer is scrapped.
Publication
Olszewski, "Generalized Feature Extraction for Structural Pattern Recognition in Time-Series Data", PhD thesis, Carnegie Mellon University, CMU-CS-01-108, 2001 (read in full from cs.cmu.edu/~bobski). Sec. 4.3.2, p. 65: "A collection of in-line process-control measurements recorded from various sensors during the processing of silicon wafers for semiconductor fabrication constitute the wafer database; each data set in the wafer database contains the measurements recorded by one sensor during the processing of one wafer by one tool." Same section, p. 66: "The data sets contained in each database were analyzed by appropriate domain experts, and a label of normal or abnormal was assigned to each data set. Of the 1194 data sets in the wafer database, 1067 data sets were identified as normal and 127 data sets were identified as abnormal." On which sensors: p. 66-67, "six parameters have been identified by domain experts as being critical for monitoring purposes: radio frequency forward power, radio frequency reflected power, chamber pressure, 405 nanometer (nm) emission, 520 nanometer (nm) emission, and direct current bias" and p. 69, "Previous experiments have shown that the most accurate classification of wafer data sets can be accomplished using the 405 nm and 520 nm parameters."
Full record
Benchmark task
Classify a length-152 etch-tool sensor trace as a normal or abnormal processing run.
Who would use it
semiconductor fabs running statistical process control on etch tools — catching an out-of-tolerance run early avoids scrapping wafers
Task construction
passnone (source task label)
The benchmark label is the target the source study posed (whether the process run recorded by the sensor was normal or abnormal, as judged by a domain expert); label origin: source task.
Data origin
temporalindex: time
Archive changes
resamplingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Archive default split, 1000 train / 6164 test; not stated in Olszewski's thesis, author and rule undocumented RR
Multichannel source
yes, one channel from a multichannel recording
Each wafer was recorded by six in-line sensors. The release ships all six traces as separate univariate series and drops the sensor identity and the wafer and run grouping, so the multivariate structure a process-control system would use is gone. The thesis itself classified single sensors (the 405 nm and 520 nm parameters) and left combining them to future work; no multivariate version is in UEA.
Dataset family
none from the same source task
Overall verdict
fails leakage
New case assumed
New wafers or production runs.
Split evidence
the archive pools all six sensors of each wafer, and 690 wafers and 99 of 104 runs have series on both sides, measured by matching every shipped series to the raw files D. The reconstruction reproduces the shipped values but is not the historical conversion code R.
WineSPECTRO · Spectroscopy & chemistry · 57/54 train/test · length 234 · 2 classesreal tasksource task labelUnclearassumed new case: new physical samplesordered, non-temporalwavelength or wavenumber
Taskreal task
Source task
real task grape variety of a red wine (Cabernet Sauvignon vs Shiraz), per the archive page
Archive task
grape variety of a red wine (Cabernet Sauvignon vs Shiraz), per the archive page
Label attached by
Unknownsource taskSR
Verdict
real task
No source study was located; the archive page names grape variety (Cabernet Sauvignon vs Shiraz) as the class; no source documents a stakeholder or use for this dataset. This rests on the archive's account alone.
SplitUnclear
New case assumed
New physical samples
Exact duplicates
none found D
Why
We do not know how many physical wines produced the spectral measurements or whether one wine was measured repeatedly. The two sets might contain measurements of the same wine. Not finding obvious similar pairs does not settle this question. Physical-sample identifiers are needed to check the split.
Source
Varietal authentication of red wine: the archive says the two classes are Cabernet Sauvignon and Shiraz and that the spectra were obtained by FTIR with attenuated total reflectance sampling, which places it in the same Institute of Food Research (Kemsley, Norwich) food-authentication family as Coffee, Meat, OliveOil and Strawbe…
Publication
Not found. The archive page gives no citation, only "donated by K. Kemsley, A. Bagnall" and a data_source_link to the IFR "Example Datasets for Download" page. Three Internet Archive snapshots of that page (2013, 2014, 2016) were retrieved: it lists strawberry purees, minced meats, olive oils, dry-cured ham SDS-PAGE lanes, coffee, DGGE lanes and facial EMG - but no wine dataset. A Europe PMC author search for Kemsley EK (58 records) returns no paper on wine. Bagnall et al. (SDM 2012) Sec. 3.1, which introduced the other IFR spectrograms to the TSC literature, covers only beef, coffee and olive oil. So the only evidence for the account above is the archive page itself.
Full record
Benchmark task
Classify a univariate length-234 mid-IR spectrum as Cabernet Sauvignon or Shiraz.
Who would use it
not documented: no source states a user or use case for this dataset
Task construction
passnone (source task label)
The benchmark label is the grape variety named on the archive page (Cabernet Sauvignon vs Shiraz), assumed to be the source's own target; no source study was located R. Label origin: source task.
Data origin
ordered, non-temporalindex: wavelength or wavenumber
Archive changes
spectrum-as-series
Leakage
unknownsource separation undocumented
Split sources
Archive default split; who built it and by what rule is not documented RDR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
real task; leakage unknown
New case assumed
New physical samples.
Split evidence
number of physical wines behind 111 spectra unknown; a nearest-neighbour check found no clear replicate pairing R+D.
real task word spotting: matching handwritten word images by word identity to index historical manuscripts
Archive task
which of 25 curator-defined word groups a word-image belongs to (the 50 words merged)
Label attached by
CuratorsredefinedS
Verdict
failstask replacement
The source study posed word-image retrieval (word spotting) to help libraries index handwritten historical collections: a real task, without classes.
SplitUnclear
New case assumed
New manuscript pages
Exact duplicates
none found D
Why
Same as FiftyWords. [FiftyWords: The word images come from one writer's manuscripts. Words on the same page can share handwriting and scanning conditions. We do not know whether whole pages were kept in one set. The test may therefore contain more words from familiar pages, rather than words from pages the model has never seen.]
Source
The same George Washington word-image collection as FiftyWords, from Rath & Manmatha's work on word spotting in handwritten historical manuscripts.
Publication
Rath & Manmatha, "Word image matching using dynamic time warping", CVPR 2003 (read) for the underlying collection and representation. The regrouping to 25 classes and the left-right flip are documented only on the archive page; no source explains which words were grouped together or on what criterion.
Full record
Benchmark task
Classify a length-270 (reversed) word-profile series into one of 25 classes.
Who would use it
libraries and archives wanting searchable indexes over handwritten collections (the 25-class regrouping has no counterpart in that use)
Task construction
failtask replacement
Benchmark target: 25 curator-defined groups of the 50 FiftyWords word types. Source target or purpose: word spotting, matching word images by their ASCII tags to index the George Washington manuscripts (Rath & Manmatha 2003, Sec. 3.1-3.2). Label attached by: Curators. Mechanism 'task replacement': the archive replaces the source's label with a different task, or defines classes by its own rule where the source posed none; here the 50 words chosen for FiftyWords (not a class set of the source) were remapped to 25 groups on an undocumented criterion (archive page: 'remapped to 25 classes').
Data origin
ordered, non-temporalindex: spatial position
Index is position along a column profile or gel lane, not time.
Archive changes
contour-unrollingresamplingz-normalization
Leakage
unknownsource separation undocumented
Split sources
Curators' split 267/638 of the same 905 instances as FiftyWords; rule undocumented RR
the same 905 word images, mirrored and re-split; WordSynonyms merges the 50 classes into 25
Overall verdict
fails task construction (leakage unknown)
New case assumed
New manuscript pages.
Split evidence
word images from one writer's manuscripts; deployment indexes new pages of the same collection; page-level split undocumented R.
Label
the source used word tags to evaluate retrieval and posed no classes S; 50 words were carved out afterwards and the archive remapped them to 25 groups S; who chose the 50 words and the grouping criterion are undocumented.
Open questions
Word retrieval replaced by closed-set classification, then 50 carved-out words merged into 25 curator groups (task replacement). Who selected the 50 words, and on what criterion were the 25 groups formed?
none Unsupervised discovery of behavioural motifs across strains (Brown et al. 2013); no classification target
Archive task
worm strain: N2 wild type or one of the mutants goa-1, unc-1, unc-38, unc-63
Label attached by
CuratorsmetadataS
Verdict
failsmetadata as label
The source collected the data but posed no classification target: Brown et al. built an unbiased dictionary of behavioural motifs, and the strain of each recorded worm was the experimental factor, not a prediction target.
SplitConcern
New case assumed
New worms or recordings
Exact duplicates
10 of 77 test series (13%) exactly match a training series D
Why
The original record describes each case as a separate worm recording. However, the value check finds 10 of 77 test cases identical to training cases. We therefore need worm and recording identifiers to resolve the disagreement between that source description and the released values.
Source
Behavioural genetics in the nematode Caenorhabditis elegans. Worm locomotion is a standard readout for gene function: mutants move differently from wild type, so quantifying movement is a way to phenotype genes.
Publication
Brown AE, Yemini EI, Grundy LJ, Jucikas T, Schafer WR, "A dictionary of behavioral motifs reveals clusters of genes affecting Caenorhabditis elegans locomotion", PNAS 110(2):791-796, 2013. Only the abstract was reached (the PNAS full text returned HTTP 403 and PubMed/EuropePMC did not render); the quoted sentence above is from that abstract. The C. elegans behavioural database (http://wormbehavior.mrc-lmb.cam.ac.uk/) is named by the archive page as the source of the traces, and the archive page reproduces the formatting description from Bagnall et al.'s bake-off paper.
Full record
Benchmark task
Classify a length-900 first-eigenworm amplitude series (one worm, ~15 minutes of crawling at 1 Hz) into one of five strains. 181 train / 77 test.
Who would use it
Geneticists and neurobiologists using C. elegans as a model organism, who need high-throughput behavioural phenotyping to link genes to behaviour.
Task construction
failmetadata as label
Benchmark target: worm strain: N2 wild type or one of the mutants goa-1, unc-1, unc-38, unc-63. Source target or purpose: Unsupervised discovery of behavioural motifs across strains (Brown et al. 2013); no classification target. Label attached by: Curators. Mechanism 'metadata as label': the archive derives the classes from a recording attribute; the strain is known before recording and was the source's experimental factor.
Data origin
temporalindex: time
Archive changes
channel-selectionresamplingz-normalization
Leakage
failexact cross-split matches (10 of 77 test series)
Split sources
181 train / 77 test, identical for Worms and WormsTwoClass; rule undocumented RDR
Multichannel source
yes, one channel from a multichannel recording
The source recorded video of a single worm crawling on an agar plate, tracked at 30 frames per second; the UCR release keeps a single channel of it. Two reductions, both large. (1) Shape: each video frame's full worm outline is collapsed to a single scalar, the amplitude along the first eigenworm - three of the four shape dimensions are discarded, so postures that differ in eigenworms 2-4 are indistinguishable here. The problem the source posed is multivariate, and its place is the multivariate archive, where it already exists as EigenWorms.
Worms and WormsTwoClass are the same traces with the same split, mutant strains collapsed in the latter; UEA EigenWorms derives from the same C. elegans behavioural database
Overall verdict
fails task construction and leakage
New case assumed
New worms or recordings.
Split evidence
each series is one worm's recording; plate or batch is not shipped R. Strain labels used as classes where the source study was unsupervised motif discovery over the same strains; the strain is recording metadata, so the task is counted as constructed S.
Open questions
The strains were the source's experimental factor, so a strain classifier is a supervised proxy for the phenotyping question the study asked; we count the label as metadata because the source posed no classification target.
none Unsupervised discovery of behavioural motifs across strains (Brown et al. 2013); no classification target
Archive task
wild type vs mutant (the four mutant strains collapsed into one class)
Label attached by
CuratorsmetadataS
Verdict
failsmetadata as label
The source collected the data but posed no classification target: Brown et al. built an unbiased dictionary of behavioural motifs, and the strain of each recorded worm was the experimental factor, not a prediction target.
SplitConcern
New case assumed
New worms or acquisitions
Exact duplicates
10 of 77 test series (13%) exactly match a training series D
Why
Same as Worms. [Worms: The original record describes each case as a separate worm recording. However, the value check finds 10 of 77 test cases identical to training cases. We therefore need worm and recording identifiers to resolve the disagreement between that source description and the released values.]
Source
Behavioural genetics in the nematode Caenorhabditis elegans. Worm locomotion is a standard readout for gene function: mutants move differently from wild type, so quantifying movement is a way to phenotype genes.
Publication
Brown AE, Yemini EI, Grundy LJ, Jucikas T, Schafer WR, "A dictionary of behavioral motifs reveals clusters of genes affecting Caenorhabditis elegans locomotion", PNAS 110(2):791-796, 2013. Only the abstract was reached (the PNAS full text returned HTTP 403 and PubMed/EuropePMC did not render); the quoted sentence above is from that abstract. The C. elegans behavioural database (http://wormbehavior.mrc-lmb.cam.ac.uk/) is named by the archive page as the source of the traces, and the archive page reproduces the formatting description from Bagnall et al.'s bake-off paper.
Full record
Benchmark task
Classify a length-900 first-eigenworm amplitude series (one worm, ~15 minutes of crawling at 1 Hz) as wild-type (N2) or mutant. 181 train / 77 test.
Who would use it
Geneticists and neurobiologists using C. elegans as a model organism, who need high-throughput behavioural phenotyping to link genes to behaviour.
Task construction
failmetadata as label
Benchmark target: wild type vs mutant (the four mutant strains collapsed into one class). Source target or purpose: Unsupervised discovery of behavioural motifs across strains (Brown et al. 2013); no classification target. Label attached by: Curators. Mechanism 'metadata as label': the archive derives the classes from a recording attribute; the strain is known before recording and was the source's experimental factor.
Data origin
temporalindex: time
Archive changes
channel-selectionresamplingz-normalization
Leakage
failexact cross-split matches (10 of 77 test series)
Split sources
181 train / 77 test, identical for Worms and WormsTwoClass; rule undocumented RDR
Multichannel source
yes, one channel from a multichannel recording
The source recorded video of a single worm crawling on an agar plate, tracked at 30 frames per second; the UCR release keeps a single channel of it. Two reductions, both large. (1) Shape: each video frame's full worm outline is collapsed to a single scalar, the amplitude along the first eigenworm - three of the four shape dimensions are discarded, so postures that differ in eigenworms 2-4 are indistinguishable here. The problem the source posed is multivariate, and its place is the multivariate archive, where it already exists as EigenWorms.
Worms and WormsTwoClass are the same traces with the same split, mutant strains collapsed in the latter; UEA EigenWorms derives from the same C. elegans behavioural database
Overall verdict
fails task construction and leakage
New case assumed
New worms or acquisitions.
Split evidence
each series is one worm's recording; plate or batch is not shipped R. Strain labels used as classes where the source study was unsupervised motif discovery over the same strains; the strain is recording metadata, so the task is counted as constructed S; the four mutant strains are collapsed into one class S.
Open questions
The strains were the source's experimental factor, so a strain classifier is a supervised proxy for the phenotyping question the study asked; we count the label as metadata because the source posed no classification target. Four mutant strains collapsed into one class.
toy task Actor discrimination, posed by the collectors as a demo task
Archive task
Which of two actors, from silhouettes
Label attached by
Collectors (= curators)instance identityS
Verdict
failsinstance identity as label · toy source task (primary: instance identity as label)
The source data were staged or generated to demonstrate a method.
SplitConcern
New case assumed
New filming occasions
Exact duplicates
none found D
Why
The task identifies two actors from video frames. Measured on the shipped files, train and test come from one pool of frames: no test series is identical to a training series, but every training frame has a near-twin in test (median 2.09 vs 23.8 for random pairs), and the training set is indistinguishable from a random draw of test frames. The test does not establish identification in footage collected on another occasion.
Source
Built by Wei & Keogh (KDD 2006, Sec. 4.5) as a "realistic dataset" for their semi-supervised time-series classification method: "The dataset was obtained by capturing two actors transiting between yoga poses in front of a green screen." The task is to tell the two actors apart - "In total, we have 316 time series with an averag…
Publication
Wei & Keogh, "Semi-Supervised Time Series Classification", KDD 2006, Sec. 4.5 (read).
Full record
Benchmark task
Classify a length-426 body-outline series as the female or the male actor.
Who would use it
none applied; the data were captured to demonstrate a semi-supervised learning method, so the "problem" is already benchmark-shaped
Task construction
failinstance identity as label · toy source task (primary: instance identity as label)
Benchmark target: Which of two actors, from silhouettes. Source target or purpose: Actor discrimination, posed by the collectors as a demo task. Label attached by: Collectors (= curators). Mechanism 'instance identity as label': the archive uses the recording's source (subject, sensor or actor) as the class. Mechanism 'toy source task': the data were simulated or staged by method developers and the classes chosen to demonstrate a method, not to answer an external question. Listed under two mechanisms; counted once, under 'instance identity as label'.
Data origin
ordered, non-temporalindex: contour arc-length
Archive changes
contour-unrollingresamplingz-normalization
Leakage
faildocumented or data-verified shared source
Split sources
Archive default split; who built it and by what rule is not documented RR
Multichannel source
no
Dataset family
none from the same source task
Overall verdict
fails task construction and leakage
New case assumed
New filming occasions.
Split evidence
measured on the shipped files, every training frame has a near-twin in test and the training set is indistinguishable from a random draw of test frames D; that the split therefore shares filming occasions is inferred from these distances, literal frame adjacency being unrecoverable R.
Label
Wei & Keogh captured the two actors themselves and posed the actor-discrimination task in KDD 2006 (collectors = curators); the series is a body outline (index = contour arc-length) S.
Open questions
Collectors posed the actor task themselves (collectors = curators); listed under both instance identity and toy source.