[Rivet] Using zipped I/O

[Andy Buckley]

I had a look this evening... it was a bit fiddly, but I've converted all
the Python write and read functions to use the C++ IO functions, and have
moved more .gz filename detection into the C++. So it works now. Needs some
improvements to converter script argument parsing, but I think should work
now for your purposes.

Andy


On 25 April 2018 at 11:38, Corpe, Louie <l.corpe at ucl.ac.uk> wrote:

> Thanks Andy,
>
> I'll give it a try.
>
> Louie
>
> On 25 Apr 2018, at 11:33, Andy Buckley <andy.buckley at cern.ch> wrote:
>
> I dug a bit more: it works from C++ but not from Python. Funny, I could
> have sworn that I had both working, but apparently not. It also doesn't
> work for gz output at the Python level, apparently: yoda2yoda test.yoda
> test.yoda.gz does not work, but it should.
>
> The Python interface to the YODA IO functions is in
> pyext/yoda/include/IO.pyx. It does have to play some games to convert the
> Pythonic view of files to the C++ stream one, and maybe this is where it's
> going wrong. It's certainly where the complaint about the ascii encoding is
> coming from. Could you take a look there and see if you can persuade it to
> do the right thing?
>
> The zipped IO itself is only supported for the YODA format, and is
> implemented in src/ReaderYODA.cc <http://readeryoda.cc/> and src/Writ
> erYODA.cc <http://writeryoda.cc/> -- it should be completely automatic
> and you shouldn't have to touch this code, but it might help for
> understanding how to interface it to the Python.
>
> Andy
>
>
> On 23 April 2018 at 13:59, Corpe, Louie <l.corpe at ucl.ac.uk> wrote:
>
>> Hi Riveters,
>>
>>
>> I'm trying to use the zipped i/o functionality introduced in Rivet 2.6.0
>> / Yoda 1.7.0.
>> I can't seem to get it working: is there an example of how to do this
>> somewhere?
>>
>> I've included a minimal example [1]. Please let me know what I'm doing
>> wrong!
>> The files I used in [1] can be found on my public on lxplus: /afs/
>> cern.ch/user/l/lcorpe/public/RivetZippedIO/examples/
>>
>> All the best and thanks in advance !
>>
>>
>> Louie
>>
>>
>> [1]
>> [lcorpe at lxplus005 scratch]$ gzip -c out1028_DSIDxxx.MEWeight.yoda >
>> examples/out1028_DSIDxxx.MEWeight.yoda.gz
>> [lcorpe at lxplus005 scratch]$ python
>> Python 2.7.5 (default, Aug  4 2017, 00:39:18)
>> [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import yoda as y
>> >>> a= y.read("examples/out1028_DSIDxxx.MEWeight.yoda.gz")
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "yoda/include/IO.pyx", line 107, in yoda.core.read (/afs/
>> cern.ch/user/l/lcorpe/scratch/rivet/YODA-1.7.0/pyext/yoda/core.cpp:85809)
>> UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 1:
>> ordinal not in range(128)
>> >>>
>> _______________________________________________
>> Rivet mailing list
>> Rivet at projects.hepforge.org
>> https://www.hepforge.org/lists/listinfo/rivet
>>
>
>
>
> --
> Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
> Particle Physics Expt Group, University of Glasgow
>
>
>


-- 
Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
Particle Physics Expt Group, University of Glasgow
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.hepforge.org/lists-archive/rivet/attachments/20180430/daeb9e19/attachment.html>