[Rivet] YODA development

[Andy Buckley]

Ben Waugh wrote:
> Hi Andy, All,
> 
> I've been having a look at the YODA pages, in particular your note on 
> the treatment of weights: 
> http://projects.hepforge.org/yoda/trac/wiki/WeightTreatment
> 
> Unless I am simply misunderstanding what you are trying to say, I'm 
> afraid I disagree with two of your three bullet points. In some cases 
> weights may allow more rapid convergence of some quantities, but they 
> occur anywhere where the generated distribution of points is not the 
> same as the desired distribution.
> 
> It is not true that "the errors must be equivalent to filling with a 
> larger number of samples with smaller weights which add up to the same". 
> ITaking it to an extreme, a bin with 100 entries each with unit weight 
> has an error of 10%, while one with a single entry with a weight of 100 
> does not have the same precision at all.
> 
> And the same principle extends to negative weights. Equal numbers of 
> large negative and positive weights indicate a correspondingly large 
> uncertainty in the value of the distribution in that bin, rather than 
> having no effect on the uncertainty at all as you suggest.
> 
> Am I wrong?

I'm not sure... I looked in the stats literature quite a lot to find a 
definitive treatment of weighted stats combination (especially where 
negative weights are concerned) and found nothing, hence the bullet 
points are my invention. Perhaps it's sensitive to how the weights are 
determined as to whether high weights are statistically equivalent to 
many small weights, but I'm inclined to agree that my bullet points 
aren't the whole story and that we need a more careful treatment. 
Fortunately, the code in YODA that handles this for all histo types is 
very localised! (in Dbn1D)

Any suggestions of alternative weighted combination recipes, which 
consistently handle positive and negative weights? Or, better, has 
anyone found any papers or books that address this issue?

Andy

-- 
Dr Andy Buckley
SUPA Advanced Research Fellow
Particle Physics Experiment Group, University of Edinburgh