<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Dear Rivet developers,<br>
<br>
As a follow-up to my mail from Monday, here is the output of my
rivet-diffhepdata-all script for Rivet 2.6.1:<br>
<br>
<a class="moz-txt-link-freetext" href="http://ippp.dur.ac.uk/~watt/RivetDiffHEPData/Rivet-2.6.1/">http://ippp.dur.ac.uk/~watt/RivetDiffHEPData/Rivet-2.6.1/</a><br>
<br>
"Of 378 Rivet analyses in ../Rivet-2.6.1/analyses, 73 (19.3%) were
compatible and 305 (80.7%) were incompatible."<br>
<br>
Compared to Rivet 2.6.0, you added 6 compatible analyses and 13
incompatible analyses, so clearly you're not yet enforcing HEPData
compatibility for new analyses. At least the percentage of
compatible analyses has improved slightly (18.7% to 19.3%). I
modified
<a class="moz-txt-link-freetext" href="http://ippp.dur.ac.uk/~watt/RivetDiffHEPData/release-2-6-x/yodadiff">http://ippp.dur.ac.uk/~watt/RivetDiffHEPData/release-2-6-x/yodadiff</a>
to print numbers to five significant figures where there are
differences. Here is a summary of the HEPData compatibility for
each of the new analyses listed at
<a class="moz-txt-link-freetext" href="http://rivet.hepforge.org/anadiffs.txt">http://rivet.hepforge.org/anadiffs.txt</a> :<br>
<br>
ALICE_2017_I1620477 : NOT compatible (different 'y')<br>
ATLAS_2012_I1094061 : compatible<br>
ATLAS_2014_I1310835 : compatible<br>
ATLAS_2016_I1502620_W : multiple Rivet analyses for one INSPIRE ID<br>
ATLAS_2016_I1502620_W_EL : multiple Rivet analyses for one INSPIRE
ID<br>
ATLAS_2016_I1502620_W_MU : multiple Rivet analyses for one INSPIRE
ID<br>
ATLAS_2016_I1502620_Z : multiple Rivet analyses for one INSPIRE ID<br>
ATLAS_2016_I1502620_Z_EL : multiple Rivet analyses for one INSPIRE
ID<br>
ATLAS_2016_I1502620_Z_MU : multiple Rivet analyses for one INSPIRE
ID<br>
ATLAS_2017_I1604029 : no HEPData record exists<br>
ATLAS_2017_I1614149 : HEPData record has extra tables<br>
ATLAS_2017_I1624693 : compatible<br>
ATLAS_2017_I1625109 : HEPData record has extra tables<br>
ATLAS_2017_I1626105 : compatible<br>
ATLAS_2017_I1644367 : NOT compatible (different 'y')<br>
ATLAS_2017_I1645627 : no HEPData record exists<br>
ATLAS_2018_I1646686 : NOT compatible<br>
CMS_2012_I1111014 : missing from analyses.json<br>
CMS_2016_I1487277 : compatible<br>
CMS_2016_I1491950 : missing from analyses.json<br>
CMS_2017_I1499471 : missing from analyses.json<br>
CMS_2017_I1605749 : missing from analyses.json<br>
CMS_2017_I1610623 : no HEPData record exists<br>
CMS_2017_I1635889 : no HEPData record exists<br>
CMS_2018_I1662081 : no HEPData record exists<br>
CMS_2018_I1663452 : no HEPData record exists<br>
CMS_2018_I1663958 : missing from analyses.json<br>
LHCF_2015_I1351909 : compatible<br>
LHCF_2016_I1385877 : NOT compatible<br>
<br>
The idea of having multiple Rivet analyses (and therefore multiple
.yoda files) for a single INSPIRE ID is really not compatible with a
HEPData download. Could the multiple .yoda files be concatenated
into one .yoda file per INSPIRE ID? Also, 5 of the new CMS analyses
appear in analyses.html but not in analyses.json and hence are not
processed by my script, so something must have gone wrong in
producing the latest analyses.json file.<br>
<br>
Best regards,<br>
Graeme<br>
<br>
<br>
<div class="moz-cite-prefix">On 13/08/18 18:24, Graeme Watt wrote:<br>
</div>
<blockquote type="cite"
cite="mid:8256e632-c81c-31a0-a1ff-02aa1cc182b8@durham.ac.uk">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
Dear Andy,<br>
<br>
Thanks for the comments. The new HEPData site already supports 1D
data types exported to YODA as a Scatter1D. Looking at the SVN
history of AidaFormatter.java in the old HepData software, I see
that you made a commit on 13th November 2009 with a message
"Adding edge guessing to AIDA formatter, for Rivet use". These
artificial bin widths are introduced not only for single-bin
measurements, e.g. there is a comment:<br>
<br>
// If there are several bins, make the internal<br>
// edges match, and the external edges duplicate<br>
// their +/- partners<br>
<br>
It would be better if these artificial bin widths were added in
the Rivet analysis code rather than in the YODA reference data,
for the cases where only the bin centres are given in HEPData
without the corresponding bin widths.<br>
<br>
I modified the latest "yodadiff" script from the hg repository to
give information on whether the 'x' or 'y' (or both) axes differ
between two YODA objects. I also added an option to compare
annotations, removed an unimplemented option to write output to a
given file, and added a check to the "eq(a, b)" function to catch
the case of "a == -b" mentioned in my last message. Can you
please include these changes in the next YODA release?<br>
<br>
<a class="moz-txt-link-freetext"
href="http://ippp.dur.ac.uk/%7Ewatt/RivetDiffHEPData/release-2-6-x/yodadiff"
moz-do-not-send="true">http://ippp.dur.ac.uk/~watt/RivetDiffHEPData/release-2-6-x/yodadiff</a><br>
<br>
I upgraded the YODA version used in the HEPData export to 1.7.0
and fixed the bug with the table index beginning at 0 or 2 instead
of 1 (as mentioned in my original message). Running my
"rivet-diffhepdata-all" script again over all analyses listed in <a
class="moz-txt-link-freetext"
href="http://rivet.hepforge.org/analyses.json"
moz-do-not-send="true">http://rivet.hepforge.org/analyses.json</a>
and taking the YODA files from the Rivet hg repository gives
output:<br>
<br>
<a class="moz-txt-link-freetext"
href="http://ippp.dur.ac.uk/%7Ewatt/RivetDiffHEPData/release-2-6-x/"
moz-do-not-send="true">http://ippp.dur.ac.uk/~watt/RivetDiffHEPData/release-2-6-x/</a><br>
<br>
The main result has only improved by one analysis since last time:<br>
<br>
"Of 359 Rivet analyses in ../rivet/analyses, 67 (18.7%) were
compatible and 292 (81.3%) were incompatible."<br>
<br>
Again, 31 of these 292 incompatible analyses are missing a HEPData
record. Grepping the .txt files in the YodaDiffOutput/ directory
for information on which axes are affected, there are 8010 data
points (across 88 analyses) with only the 'x' axis affected, 7930
data points (across 100 analyses) with only the 'y' axis affected,
and 5791 data points (across 50 analyses) with both 'x' and 'y'
axes affected. Can you please include the "rivet-diffhepdata" and
"rivet-diffhepdata-all" scripts in the bin/ directory of the next
Rivet release?<br>
<br>
<a class="moz-txt-link-freetext"
href="https://github.com/HEPData/miscellaneous/blob/master/scripts/rivet-diffhepdata"
moz-do-not-send="true">https://github.com/HEPData/miscellaneous/blob/master/scripts/rivet-diffhepdata</a><br>
<a class="moz-txt-link-freetext"
href="https://github.com/HEPData/miscellaneous/blob/master/scripts/rivet-diffhepdata-all"
moz-do-not-send="true">https://github.com/HEPData/miscellaneous/blob/master/scripts/rivet-diffhepdata-all</a><br>
<br>
Is the "rivet-diffhepdata" script already being used as part of
the validation procedure for analyses added to new Rivet
releases? If not, it would be good to build it into your
validation machinery. Can you please also modify the
"rivet-mkanalysis" script to pass the Rivet analysis name as an
argument to the HEPData download URL as I requested below?<br>
<br>
hdurl = <a class="moz-txt-link-rfc2396E"
href="http://www.hepdata.net/record/ins%s?format=yoda&rivet=%s"
moz-do-not-send="true"><font color="red"><b>MailScanner has detected a possible fraud attempt from "www.hepdata.net" claiming to be</b></font> "http://www.hepdata.net/record/ins%s?format=yoda&rivet=%s"</a>
% (ANAINSPIREID, ANANAME)<br>
<br>
Best regards,<br>
Graeme<br>
<br>
P.S. Have you seen <a class="moz-txt-link-freetext"
href="https://github.com/hdembinski/histogram"
moz-do-not-send="true">https://github.com/hdembinski/histogram</a>
(maybe an alternative to YODA if it becomes part of Boost)?<br>
<br>
<br>
<div class="moz-cite-prefix">On 18/06/18 14:31, Andy Buckley
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:1529317424.local-df3b6909-a71c-v1.2.2-96fb3a99@getmailspring.com">
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
<div>Hi Graeme,</div>
<br>
<div>Sorry to be coming late to this.</div>
<br>
<div>I absolutely agree that we should get rid of the fake "unit
bin width" for unbinned quantities -- that was an ad hoc
solution from deep in prehistory to HepData's fundamental data
type being binned, and now that's not the case we should move
away from it. Writing out HepData 1D data types to YODA as a
Scatter1D would be best (and then, as Jon as noted before, we
just need to make sure that those get plotted in a reasonable
way).</div>
<br>
<div>The level of disagreement between HepData and Rivet records
is staggering, and I'm sorry that we've not had the manpower
to deal with that. Holger was previously working on it, and I
think found similar numbers, but then moved institute (to
FNAL) before he could act on it; maybe there is still some
possibility to follow it up, Holger?</div>
<br>
<div>Since you have diagnostics scripts operational at the
moment, Graeme, could you produce a list/summary of how many
records only differ by the <em>binning</em> of observables,
and maybe another with a <em>loose</em> consistency check on
bin contents? That would give us some insight into how many
differ because the Rivet files "just" need to be synced with
HepData, and how many have completely inconsistent dataset
naming. The latter are the ones that will need some analysis
code updates.</div>
<br>
<div>I appreciate that this task maybe falls into a gap between
purely clerical and something requiring physics insight.
However, if we can manage to isolate a purely clerical
component then help from Joanna would be really valuable: as
you're aware we have no dedicated manpower whatsoever on the
Rivet side, and tasks of this sort are the hardest to motivate
postdocs to work on. I am also pessimistic that we can get
help from the LHC experiments (who supplied us with the
inconsistent Rivet/HD records in the first place...), but we
can try: breakdown of the numbers by experiment would also be
very useful.</div>
<br>
<div>Thanks,</div>
<div>Andy</div>
<br>
<div><signature id="initial">
<div>
<table cellspacing="0" cellpadding="0">
<tbody>
<tr>
<td colspan="2">
<div style="padding-bottom:15px;">
<div><strong>Dr Andy Buckley, Lecturer / Royal
Society University Research Fellow</strong></div>
<div>Particle Physics Experiment Group,
University of Glasgow</div>
</div>
</td>
</tr>
<tr>
<td style="vertical-align:top;"><br>
</td>
<td><br>
</td>
</tr>
</tbody>
</table>
</div>
</signature></div>
<div class="gmail_quote_attribution">On Jun 15 2018, at 2:20 pm,
Graeme Watt <a class="moz-txt-link-rfc2396E"
href="mailto:Graeme.Watt@durham.ac.uk"
moz-do-not-send="true"><Graeme.Watt@durham.ac.uk></a>
wrote:</div>
<blockquote><br>
<div>
<div> Dear Peter (and Rivet developers),</div>
<br>
<div>No, I don't think this is something that Joanne (or
Keith) could help with.</div>
<br>
<div><a
href="https://github.com/HEPData/miscellaneous/blob/master/scripts/rivet-diffhepdata-all"
title="https://github.com/HEPData/miscellaneous/blob/master/scripts/rivet-diffhepdata-all"
moz-do-not-send="true">https://github.com/HEPData/miscellaneous/blob/master/scripts/rivet-diffhepdata-all</a></div>
<br>
<div>I wrote another script "rivet-diffhepdata-all" that
loops over all Rivet analyses listed in <a
href="http://rivet.hepforge.org/analyses.json"
title="http://rivet.hepforge.org/analyses.json"
moz-do-not-send="true">http://rivet.hepforge.org/analyses.json</a>
and compares each Rivet .yoda file with the HEPData
download. It calls functions from the previous
"rivet-diffhepdata" script which in turn calls "yodadiff".</div>
<br>
<div>I added the URL option to pass the Rivet analysis name
to HEPData when requesting a YODA conversion, e.g.</div>
<br>
<div><a
href="https://hepdata.net/record/ins319520?format=yoda&rivet=ALEPH_1991_S2435284"
title="https://hepdata.net/record/ins319520?format=yoda&rivet=ALEPH_1991_S2435284"
moz-do-not-send="true">https://hepdata.net/record/ins319520?format=yoda&rivet=ALEPH_1991_S2435284</a></div>
<br>
<div>This is necessary, for example, for Rivet analysis
names containing the SPIRES ID rather than the INSPIRE
ID. The "rivet-mkanalysis" script could be modified
accordingly:</div>
<br>
<div><a
href="https://rivet.hepforge.org/trac/browser/bin/rivet-mkanalysis#L139"
title="https://rivet.hepforge.org/trac/browser/bin/rivet-mkanalysis#L139"
moz-do-not-send="true">https://rivet.hepforge.org/trac/browser/bin/rivet-mkanalysis#L139</a></div>
<div>hdurl = <a
href="http://www.hepdata.net/record/ins%s?format=yoda&rivet=%s"
title="http://www.hepdata.net/record/ins%s?format=yoda&rivet=%s"
moz-do-not-send="true"><span style="color:red"><strong>MailScanner
has detected a possible fraud attempt from
"www.hepdata.net" claiming to be</strong></span></a><a
href="http://www.hepdata.net/record/ins%s?format=yoda&rivet=%s"
title="http://www.hepdata.net/record/ins%s?format=yoda&rivet=%s"
moz-do-not-send="true"><font color="red"><b>MailScanner
has detected a possible fraud attempt from
"www.hepdata.net" claiming to be</b></font>
"http://www.hepdata.net/record/ins%s?format=yoda&rivet=%s"</a>
% (ANAINSPIREID, ANANAME)</div>
<br>
<div>The web directory <a
href="http://ippp.dur.ac.uk/%7Ewatt/RivetDiffHEPData/Rivet-2.6.0/"
title="http://ippp.dur.ac.uk/%7Ewatt/RivetDiffHEPData/Rivet-2.6.0/"
moz-do-not-send="true">http://ippp.dur.ac.uk/~watt/RivetDiffHEPData/Rivet-2.6.0/</a>
contains the output of running:</div>
<br>
<div>rivet-diffhepdata-all -r ../Rivet-2.6.0/analyses -d
HEPDataYoda -o YodaDiffOutput >
rivet-diffhepdata-all.txt</div>
<br>
<div>The summary line doesn't look too promising:</div>
<br>
<div>"Of 359 Rivet analyses in ../Rivet-2.6.0/analyses, 66
(18.4%) were compatible and 293 (81.6%) were
incompatible."</div>
<br>
<div>Here, compatibility is defined as a zero exit status
returned by "yodadiff". Only 31 of these 293 incompatible
analyses are missing a HEPData record.</div>
<br>
<div>Note that the "yodadiff" script gives a
ZeroDivisionError in the function "eq(a, b)" when "a" and
"b" have opposite sign due to the return value:</div>
<br>
<div>return abs(float(a) - float(b))/(float(a) + float(b))
< opts.TOL</div>
<br>
<div>For example, d01-x01-y01 of ATLAS_2013_I1190187.yoda
distributed with Rivet has yerr- = 6.8 and yerr+ = -6.8.
The HEPData table ( <a
href="http://www.hepdata.net/record/ins1190187?version=1&table=Table1"
title="http://www.hepdata.net/record/ins1190187?version=1&table=Table1"
moz-do-not-send="true">http://www.hepdata.net/record/ins1190187?version=1&table=Table1</a>
) gives both yerr- and yerr+ as 1.212930e+01, so it seems
that the Rivet .yoda file contains only the statistical
error (with a wrong sign for yerr+). The Rivet .yoda file
also assigns an artificial bin width of 1 for sqrt(s),
whereas the HEPData table does not assign a bin width for
sqrt(s). Looking at the YODA export from the old HepData
site:</div>
<br>
<div><a
href="http://hepdata.cedar.ac.uk/view/ins1190187/d1/yoda"
title="http://hepdata.cedar.ac.uk/view/ins1190187/d1/yoda"
moz-do-not-send="true">http://hepdata.cedar.ac.uk/view/ins1190187/d1/yoda</a></div>
<br>
<div>again there are zero xerr- and xerr+ values, but the
AIDA export:</div>
<br>
<div><a
href="http://hepdata.cedar.ac.uk/view/ins1190187/d1/aida"
title="http://hepdata.cedar.ac.uk/view/ins1190187/d1/aida"
moz-do-not-send="true">http://hepdata.cedar.ac.uk/view/ins1190187/d1/aida</a></div>
<br>
<div>has a unit bin width written by "AidaFormatter.java"
with a comment (by Andy?):</div>
<br>
<div>// If there's only one bin and it has no width, give</div>
<div>// it unit width so that it can be filled and the
height</div>
<div>// doesn't go mad when Rivet tries to use it.</div>
<br>
<div>I would argue that any construction of artificial bin
widths would be better handled on the Rivet side rather
than in the HEPData export of the data. Removing
artificial bin widths from the Scatter objects in the
Rivet .yoda files would resolve some of the
incompatibilities between HEPData and Rivet, but many more
would remain. I don't propose a universal solution for
now, but just want to start by identifying where the
differences lie. It will be interesting to monitor
whether the degree of incompatibility, now easily
quantified by my new "rivet-diffhepdata-all" script,
improves for subsequent Rivet releases.</div>
<br>
<div>Best regards,</div>
<div>Graeme Watt (HEPData)</div>
<br>
<br>
<div>
<div>On 25/05/18 23:17, Peter Skands wrote:</div>
</div>
<blockquote><br>
<div><font style="font-size:13px"><font
style="font-family:Helvetica, Arial">Hi All,</font></font></div>
<div><br>
</div>
<div><font style="font-size:13px"><font
style="font-family:Helvetica, Arial">Agree it sounds
like it could make sense as part of Rivet, and could
make a real difference by making the step to check
consistency almost trivial; easy to run when
developing / releasing analyses. In the latter case,
feedback could go back to the people submitting a
new analysis if there is a discrepancy, which would
then put the burden of ensuring consistency mainly
on the people who contribute the analyses, without
adding significantly to the Rivet / HepData authors.
Regarding the existing / old analyses, Graeme, any
chance you think of the person in Durham having a go
at the backlog of old analyses, or would that be too
challenging for her? It might be worth preparing an
example, show it to Keith, and see what he thinks?</font></font></div>
<div><br>
</div>
<div><font style="font-size:13px"><font
style="font-family:Helvetica, Arial">Cheers,</font></font></div>
<div><font style="font-size:13px"><font
style="font-family:Helvetica, Arial">Peter</font></font></div>
<div><br>
</div>
<div>
<div><font style="font-family:Helvetica"><font
style="font-size:11px">—</font></font></div>
<div>
<div><font style="font-family:Helvetica, Arial"><strong>PETER
SKANDS</strong></font></div>
<div><font style="font-family:Helvetica, Arial"><font
style="font-size:11px"><font
style="font-family:Helvetica">Associate
Professor</font></font></font></div>
<br>
<div><font style="font-family:Helvetica, Arial"><strong>School
of Physics and Astronomy</strong></font></div>
<div><font style="font-family:Helvetica, Arial"><font
style="font-size:11px"><font
style="font-family:Helvetica">Monash
University </font></font></font></div>
<div><font style="font-family:Helvetica, Arial"><font
style="font-size:11px"><font
style="font-family:Helvetica">10 College Walk,
Clayton Campus</font></font></font></div>
<div><font style="font-family:Helvetica, Arial"><font
style="font-size:11px"><font
style="font-family:Helvetica">Melbourne, VIC
3800</font></font></font></div>
<div><font style="font-family:Helvetica, Arial"><font
style="font-size:11px"><font
style="font-family:Helvetica">Australia</font></font></font></div>
<br>
<div><font style="font-family:Helvetica, Arial"><font
style="font-size:11px"><font
style="font-family:Helvetica">T: </font></font></font><font
style="font-family:Helvetica, Arial"><font
style="font-size:11px"><font
style="font-family:Helvetica"><a
href="tel://+61%203%20990%2053692"
title="tel://+61%203%20990%2053692"
moz-do-not-send="true">+61 3 990 53692</a></font></font></font></div>
<div><font style="font-family:Helvetica, Arial"><font
style="font-size:11px"><font
style="font-family:Helvetica">E: </font></font></font><font
style="font-family:Helvetica, Arial"><font
style="font-size:11px"><font
style="font-family:Helvetica"><a
href="mailto:peter.skands@monash.edu"
title="mailto:peter.skands@monash.edu"
moz-do-not-send="true">peter.skands@monash.edu</a></font></font></font></div>
<div><font style="font-family:Helvetica, Arial"><font
style="font-size:11px"><font
style="font-family:Helvetica">W: </font></font></font><font
style="font-family:Helvetica, Arial"><font
style="font-size:11px"><font
style="font-family:Helvetica"><a
href="http://skands.physics.monash.edu/"
title="http://skands.physics.monash.edu/"
moz-do-not-send="true">skands.physics.monash.edu</a></font></font></font></div>
</div>
</div>
<br>
<div>On 26 May 2018 at 2:52:23 am, Graeme Watt (<a
href="mailto:graeme.watt@durham.ac.uk"
title="mailto:graeme.watt@durham.ac.uk"
moz-do-not-send="true">graeme.watt@durham.ac.uk</a>)
wrote:</div>
<br>
<blockquote>
<div>
<div>
<div>Dear David,</div>
<br>
<div>OK, here's another much simpler Python script
that downloads the YODA</div>
<div>file from HEPData and then calls yodadiff:</div>
<br>
<div><a
href="https://github.com/HEPData/miscellaneous/blob/master/scripts/rivet-diffhepdata"
title="https://github.com/HEPData/miscellaneous/blob/master/scripts/rivet-diffhepdata"
moz-do-not-send="true">https://github.com/HEPData/miscellaneous/blob/master/scripts/rivet-diffhepdata</a></div>
<br>
<div>I'm not sure if this even warrants a script,
given that the</div>
<div>functionality of:</div>
<br>
<div> rivet-diffhepdata ATLAS_2017_I1614149.yoda -i
1614149</div>
<br>
<div>could be obtained with two commands, e.g.</div>
<br>
<div> curl -L <a
href="https://hepdata.net/record/ins1614149?format=yoda"
title="https://hepdata.net/record/ins1614149?format=yoda"
moz-do-not-send="true">https://hepdata.net/record/ins1614149?format=yoda</a>
| tar zx</div>
<div> yodadiff ATLAS_2017_I1614149.yoda
HEPData-ins1614149-v2-yoda.yoda</div>
<br>
<div>The yodadiff script gives more detailed output
of differences than my</div>
<div>previous script, but it does not compare
annotations (maybe this</div>
<div>functionality could be added to yodadiff as an
option?). Also, the</div>
<div>yodadiff script flags additional analysis
objects (like covariance</div>
<div>matrices) that are present in the HEPData YODA
file but not in the Rivet</div>
<div>YODA file (as is the case for
ATLAS_2017_I1614149), whereas my previous</div>
<div>script was specifically written to ignore these
differences. I suppose</div>
<div>this is OK if the user just ignores the
warnings from yodadiff about</div>
<div>additional analysis objects in the HEPData YODA
file.</div>
<br>
<div>Best regards,</div>
<div>Graeme</div>
<br>
<br>
<div>On 25/05/18 11:01, David Grellscheid wrote:</div>
<div>> Hi Graeme,</div>
<div>></div>
<div>> Yes, I did mean yodadiff, sorry! There's
no way it will start to include</div>
<div>> a Hepdata download option, it is meant to
do one job of comparing two</div>
<div>> yoda files, which may have nothing at all
to do with HEP or Rivet.</div>
<div>></div>
<div>> The download option could be provided (in
rivet/bin, not yoda/bin!) by a</div>
<div>> thin layer over the top of yoadiff,
though. If you'd like to adapt your</div>
<div>> script in that way, we'd be happy to
include it in the rivet distribution.</div>
<div>></div>
<div>> See you,</div>
<div>></div>
<div>> David</div>
<div>></div>
<div>></div>
<div>> On 22/05/2018 12:20, Graeme Watt wrote:</div>
<div>>> Dear David,</div>
<div>>></div>
<div>>> Thanks, I think you mean yodadiff (not
yodacmp). You're right, it looks</div>
<div>>> like yodadiff does much the same as my
script, other than the HEPData</div>
<div>>> download and optional comparison of
annotations. Maybe these features</div>
<div>>> could be added to yodadiff? In any
case, it would be good to make such</div>
<div>>> comparisons part of the validation and
release procedure of each new</div>
<div>>> Rivet analysis, which is apparently
not happening at the moment (other</div>
<div>>> than possibly for ATLAS analyses).</div>
<div>>></div>
<div>>> Best regards,</div>
<div>>> Graeme</div>
<div>>></div>
<div>>></div>
<div>>> On 22/05/18 09:54, David Grellscheid
wrote:</div>
<div>>>> Hi Graeme,</div>
<div>>>></div>
<div>>>> thanks for your email, we have
started discussing it. Just one technical</div>
<div>>>> point, YODA comes with a yodacmp
script already, which addresses many of</div>
<div>>>> your technical issues with
comparisons. Maybe you can use that</div>
<div>>>> internally in your script?</div>
<div>>>></div>
<div>>>> See you,</div>
<div>>>></div>
<div>>>> David</div>
<div>>>></div>
<div>>>></div>
<div>>>> On 21/05/2018 20:16, Graeme Watt
wrote:</div>
<div>>>>> Dear Rivet developers,</div>
<div>>>>></div>
<div>>>>> I wrote a Python script to
compare a YODA reference data file, intended</div>
<div>>>>> for inclusion in Rivet, with
the corresponding YODA file downloaded from</div>
<div>>>>> HEPData:</div>
<div>>>>></div>
<div>>>>> <a
href="https://github.com/HEPData/miscellaneous/blob/master/scripts/yoda_compare_hepdata.py"
title="https://github.com/HEPData/miscellaneous/blob/master/scripts/yoda_compare_hepdata.py"
moz-do-not-send="true">https://github.com/HEPData/miscellaneous/blob/master/scripts/yoda_compare_hepdata.py</a></div>
<div>>>>></div>
<div>>>>></div>
<div>>>>> Example usage:
./yoda_compare_hepdata.py ATLAS_2017_I1614149.yoda
-i</div>
<div>>>>> 1614149 -a</div>
<div>>>>></div>
<div>>>>> This means: compare a local
YODA file "ATLAS_2017_I1614149.yoda" with a</div>
<div>>>>> YODA file downloaded from the
HEPData record with INSPIRE ID "1614149"</div>
<div>>>>> and also compare YODA
annotations "-a". Since the HEPData YODA file</div>
<div>>>>> might contain additional
analysis objects compared to the Rivet YODA</div>
<div>>>>> file, and since there might be
inconsequential rounding errors or</div>
<div>>>>> differences in number formats,
comparison using a simple "diff" of .yoda</div>
<div>>>>> files is not always adequate.</div>
<div>>>>></div>
<div>>>>> I had a few problems with the
YODA 1.7.0 software when writing the</div>
<div>>>>> Python script, which could
perhaps be improved in future:</div>
<div>>>>></div>
<div>>>>> * Calling dump() on Scatter
objects does not output the same format as</div>
<div>>>>> in the input .yoda files, e.g.
dumping a Scatter2D gives "HISTO1D" in</div>
<div>>>>> the output and the central
value of the x bin is not output. This might</div>
<div>>>>> be due to deficiencies in</div>
<div>>>>> <a
href="https://yoda.hepforge.org/trac/browser/src/WriterFLAT.cc"
title="https://yoda.hepforge.org/trac/browser/src/WriterFLAT.cc"
moz-do-not-send="true">https://yoda.hepforge.org/trac/browser/src/WriterFLAT.cc</a>
.</div>
<div>>>>> * I expected to be able to
check (fuzzy) equality of two Scatter objects</div>
<div>>>>> using "s == s1", which seems
to be implemented in C++ but not in Python.</div>
<div>>>>> * Checking (fuzzy) equality of
two Point2D objects using "p == p1" is</div>
<div>>>>> implemented in Python, but it
only compares the x axes and not the y</div>
<div>>>>> axes. Similarly, for Point3D
objects, the (fuzzy) equality operator</div>
<div>>>>> "==" only compares the x and y
axes, but not the z axes.</div>
<div>>>>> * In the end, I just copied
the definition of "fuzzyEquals" from the C++</div>
<div>>>>> code into my script and did my
own comparisons, without relying on the</div>
<div>>>>> "==" operator for Point or
Scatter objects.</div>
<div>>>>></div>
<div>>>>> Recall that Holger Schulz made
some similar comparisons in 2016 between</div>
<div>>>>> YODA reference data files from
Rivet and from the old HepData, where he</div>
<div>>>>> found significant
inconsistencies:</div>
<div>>>>></div>
<div>>>>> <a
href="https://www.hepforge.org/lists-archive/rivet/2016-October/007318.html"
title="https://www.hepforge.org/lists-archive/rivet/2016-October/007318.html"
moz-do-not-send="true">https://www.hepforge.org/lists-archive/rivet/2016-October/007318.html</a></div>
<div>>>>> <a
href="https://rivet.hepforge.org/trac/browser/contrib/devscripts/HepDataConsistency"
title="https://rivet.hepforge.org/trac/browser/contrib/devscripts/HepDataConsistency"
moz-do-not-send="true">https://rivet.hepforge.org/trac/browser/contrib/devscripts/HepDataConsistency</a></div>
<div>>>>></div>
<div>>>>></div>
<div>>>>></div>
<div>>>>> While fixing all Rivet/HEPData
inconsistencies is probably unrealistic</div>
<div>>>>> for now, we could at least
start by ensuring that analyses added to new</div>
<div>>>>> Rivet releases include a YODA
file that's compatible with the YODA</div>
<div>>>>> download given by HEPData. My
new script should be useful for these</div>
<div>>>>> purposes. (This work was
prompted by a conversation with Peter Skands</div>
<div>>>>> and Jon Butterworth last month
in Durham.) You're welcome to modify my</div>
<div>>>>> script as you need and include
it in a future Rivet release. The script</div>
<div>>>>> could be run by the experiment
contact persons before they upload new</div>
<div>>>>> analyses to the Rivet
"contrib" area. It could also be run (perhaps in</div>
<div>>>>> a more automated way) by the
Rivet developers when moving analyses from</div>
<div>>>>> the "contrib" area to a new
Rivet release. I just ran the script for</div>
<div>>>>> all the new analyses in the
current Rivet "contrib" area (</div>
<div>>>>> <a
href="https://www.hepforge.org/archive/rivet/contrib/"
title="https://www.hepforge.org/archive/rivet/contrib/"
moz-do-not-send="true">https://www.hepforge.org/archive/rivet/contrib/</a>
) and it already turned</div>
<div>>>>> up some useful information:</div>
<div>>>>></div>
<div>>>>> * ATLAS_2014_I1310835,
ATLAS_2017_I1614149, and ATLAS_2017_I1624693, are</div>
<div>>>>> all compatible with HEPData.</div>
<div>>>>> * ATLAS_2016_I1502620 has
multiple .yoda files which can't be handled by</div>
<div>>>>> my script.</div>
<div>>>>> * ATLAS_2017_I1625109 showed
some apparent inconsistencies but the</div>
<div>>>>> problem looks to be on the
HEPData side. The dataset index starts at 0</div>
<div>>>>> (instead of 1) due to a bug in
the hepdata-converter package (which I'll</div>
<div>>>>> fix) and the year in the
analysis name is 2018 (instead of 2017) taken</div>
<div>>>>> from the journal publication.</div>
<div>>>>> * ALICE_2017_I1620477 is
incompatible with HEPData.</div>
<div>>>>> * CMS_2016_I1487277 is
compatible with HEPData.</div>
<div>>>>> * CMS_2012_I1111014 and
CMS_2016_I1491950 are incompatible with HEPData.</div>
<div>>>>> * CMS_2017_I1499471,
CMS_2017_I1635889, CMS_2018_I1662081 and</div>
<div>>>>> CMS_2018_I1663958 are all
missing from HEPData.</div>
<div>>>>> * CMS_2017_I1605749 has a
HEPData record, but the YODA conversion fails</div>
<div>>>>> due to a problem with the
original HEPData submission.</div>
<div>>>>> * LHCF_2015_I1351909 is
compatible with HEPData, but the annotations</div>
<div>>>>> differ.</div>
<div>>>>> * LHCF_2016_I1385877 is
incompatible with HEPData.</div>
<div>>>>></div>
<div>>>>> I hope this gives you
something to discuss at this week's Rivet</div>
<div>>>>> workshop! :-)</div>
<div>>>>></div>
<div>>>>> Best regards,</div>
<div>>>>> Graeme Watt (HEPData)</div>
<div>>>>></div>
<div>>>>></div>
<div>>>>></div>
<div>>>>></div>
<div>>>>>
_______________________________________________</div>
<div>>>>> Rivet mailing list</div>
<div>>>>> <a
href="mailto:Rivet@projects.hepforge.org"
title="mailto:Rivet@projects.hepforge.org"
moz-do-not-send="true">Rivet@projects.hepforge.org</a></div>
<div>>>>> <a
href="https://www.hepforge.org/lists/listinfo/rivet"
title="https://www.hepforge.org/lists/listinfo/rivet"
moz-do-not-send="true">https://www.hepforge.org/lists/listinfo/rivet</a></div>
<div>>>>></div>
<br>
</div>
</div>
</blockquote>
</blockquote>
<br>
<div>_______________________________________________</div>
<div>Rivet mailing list</div>
<div><a class="moz-txt-link-abbreviated"
href="mailto:Rivet@projects.hepforge.org"
moz-do-not-send="true">Rivet@projects.hepforge.org</a></div>
<div><a class="moz-txt-link-freetext"
href="https://www.hepforge.org/lists/listinfo/rivet"
moz-do-not-send="true">https://www.hepforge.org/lists/listinfo/rivet</a></div>
</div>
</blockquote>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>