|
[Rivet] ruvet and osxJames William Monk james.william.monk at cern.chMon Aug 20 16:15:09 BST 2018
Hi everyone, I have now had a chance to test the 2.6.X branch installation, and the problem persists also with the vendor supplied toolchain (though possibly a different problem to the HomeBrew below). I cannot understand how this was not an issue with 2.5 though cheers, James On 20 Aug 2018, at 11:55, Christian Gutschow <chris.g at cern.ch<mailto:chris.g at cern.ch>> wrote: Hi, sorry for having to come back to this so late. The removal of the additional ${PYTHON} is breaking the installation for Macs using Homebrew it seems (at least it did for me), since Homebrew is making its own Python installation the default. :/ Cheers, Chris On 30 Jul 2018, at 21:33, Andy Buckley <andy.buckley at cern.ch<mailto:andy.buckley at cern.ch>> wrote: Is it relevant that the doc builds are explicitly called as $(PYTHON) $(srcdir)/mk-analyses-html etc., rather than just $(srcdir)/mk-analyses-html ? Maybe the implicit "shebang" execution is not such a problem... but I don't have a Mac to test on. It works fine on my Linux system without the $(PYTHON), though, so I've changed it to that and pushed the release-2-6-x head. Really looking forward to getting past all these technical compatibility issues that crop up every time we approach this release! Andy Dr Andy Buckley, Lecturer / Royal Society University Research Fellow Particle Physics Experiment Group, University of Glasgow On Jul 30 2018, at 8:18 pm, James William Monk <james.william.monk at cern.ch<mailto:james.william.monk at cern.ch>> wrote: Thanks David, How then does the rivet main executable work then? That is itself a python script that loads the Rivet libs. How does it manage to inherit the DYLD_LIBRARY_PATH cheers, James On 30 Jul 2018, at 17:28, David Grellscheid <david.grellscheid at durham.ac.uk<mailto:david.grellscheid at durham.ac.uk>> wrote: You've diagnosed it right. All programs in /usr/bin are considered protected, and will not be passed any DYLD environment. A python in /usr/local or /opt will work fine Also, making the script executable does not help. The first line with the #! still calls either /usr/bin/env or /usr/bin/python with exactly the same issue David (on house move teabreak) On 30 July 2018 16:17:18 BST, James William Monk <james.william.monk at cern.ch<mailto:james.william.monk at cern.ch>> wrote: Hi everyone, Did we ever come across the problem that the mac System Integrity Protection (SIP) does not pass on the DYLD_LIBRARY_PATH to a subprocess if it thinks that subprocess is a “protected” process? I believe that is the problem here, and is preventing the library path being set correctly during the Rivet build because it spawns a python subprocess that does not inherit the correct environment It seems that if I do a python script to simply print the environment, i.e. import os for key in os.environ.keys(): print key then execute foo=“bar” python test (where test contains the script above) Then I see that foo is detected in the environment. However, if I do DYLD_LIBRARY_PATH=“bar” python test Then DYLD_LIBRARY_PATH does not show up! Presumably SIP has decided that the python subprocess must be protected from malicious setting of the environment. I can see why they might think that is important, since changing the path would potentially cause it to load a bad version of some library. I can disable SIP to confirm this, but that requires a reboot of my mac and is not something we should ask users to do. I think that the fix for this is that the mk-analysis-html script has to be executable. That way it will inherit from the build environment, not from the python subprocess whose environment has been wiped clean. cheers, James On 30 Jul 2018, at 16:36, James William Monk <james.william.monk at cern.ch<mailto:james.william.monk at cern.ch>> wrote: Hi Sabah, There is a bug in the build system for the docs in Rivet 2.6 on mac. This issue has occurred before, but I only just remembered it. It looks like David Grellscheid has tried to fix it in the past, but when I use the updated Makefile.am from the doc dir, I still get the same problem. I knew there was a reason I was still on Rivet 2.5.4 :) It looks to me like there is a mistake in the setup for the environment needed for the python script mk-analysis-html. At least one error is that it seems to have the PYTHONPATH pointing to the installation prefix, but there is nothing there of course because Rivet hasn’t been installed yet. Another error is that DYLD_LIBRARY_PATH is not set. I tested this by modifying doc/mk-analysis-html by adding print "Hello from mk-analysis-html. I think $PYTHONPATH = " + os.environ['PYTHONPATH'] print "environ keys are" for key in os.environ.keys(): print key This problem is only related to the building of the documentation, but disabling doxygen doesn’t seem to help. I *think* if you plough on regardless and do make install, you will get Rivet installed in your prefix, just without the docs and the rivetenv.sh script. cheers, James On 28 Jul 2018, at 13:11, Sabah Salih <sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>> wrote: Dear James, I am very grateful for your kind help I tried to apply rivet-bootstrap content line by line via command line YODA, HEPMC,... are fine When I reached Rivet I get the following: I attached the file which have the complete steps for RIVET Many Thanks, Sabah. File "/private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/__init__.py", line 12, in <module> from rivet.core import * ImportError: dlopen(/private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so, 10): Library not loaded: /tmp/rivet/local/lib/libRivet.dylib Referenced from: /private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so Reason: image not found make[1]: *** [analyses.json] Error 1 make[1]: *** Waiting for unfinished jobs.... Using output file name 'analyses.html' and directory 'analyses' Traceback (most recent call last): File "mk-analysis-html", line 25, in <module> import rivet File "/private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/__init__.py", line 12, in <module> from rivet.core import * ImportError: dlopen(/private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so, 10): Library not loaded: /tmp/rivet/local/lib/libRivet.dylib Referenced from: /private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so Reason: image not found make[1]: *** [analyses.html] Error 1 make: *** [all-recursive] Error 1 ls -lh /private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/ total 312 -rw-r--r-- 1 admin wheel 351B 3 Oct 2017 __init__.py -rw-r--r-- 1 admin wheel 598B 28 Jul 11:26 __init__.pyc -rw-r--r-- 1 admin wheel 3.0K 6 Nov 2017 aopaths.py -rwxr-xr-x 1 admin wheel 121K 28 Jul 11:15 core.so -rw-r--r-- 1 admin wheel 7.8K 3 Oct 2017 plotinfo.py -rw-r--r-- 1 admin wheel 2.5K 3 Oct 2017 spiresbib.py -rw-r--r-- 1 admin wheel 6.9K 20 Nov 2017 util.py Many Thanks, Sabah. From Sabah Salih Particle Physics Group, The School of Physics and Astronomy, The University of Manchester, Schuster Laboratory, Brunswick Street, Manchester M13 9PL. Tel: +44 1612754171 or x4171 E-mail: sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk> ________________________________________ From: James William Monk [james.william.monk at cern.ch<mailto:james.william.monk at cern.ch>] Sent: 25 July 2018 22:46 To: Sabah Salih Cc: James William Monk; Andy Buckley; rivet at projects.hepforge.org<mailto:rivet at projects.hepforge.org> Subject: Re: [Rivet] ruvet and osx Hi Sabah, That error is just confirming that the Rivet lib is not present, I presume because the build and/or installation failed. Do you see any other error that looks like a build error? I’m not very familiar with the bootstrap, so I can’t say whether it produces any build logs, or where they would be, but we need to see the build failure message to know why the libraries weren't built properly. cheers, James On 25 Jul 2018, at 19:09, Sabah Salih <sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>> wrote: Dear James, Thank you so much. Following your advice I did the following: edit rivet-bootstrap and changed function conf { ./configure --prefix=$INSTALL_PREFIX "$@"; } to function conf { ./configure CC=/usr/bin/clang CXX=/usr/bin/clang++ --prefix=$INSTALL_PREFIX "$@"; } and ./configure --prefix=$INSTALL_PREFIX to ./configure CC=/usr/bin/clang CXX=/usr/bin/clang++ --prefix=$INSTALL_PREFIX When I type ./rivet-bootstrap I get this File "/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/__init__.py", line 12, in <module> from rivet.core import * ImportError: dlopen(/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so, 10): Library not loaded: /Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/local/lib/libRivet.dylib Referenced from: /Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so Reason: image not found make[1]: *** [analyses.json] Error 1 make[1]: *** Waiting for unfinished jobs.... Using output file name 'analyses.html' and directory 'analyses' Traceback (most recent call last): File "mk-analysis-html", line 25, in <module> import rivet File "/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/__init__.py", line 12, in <module> from rivet.core import * ImportError: dlopen(/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so, 10): Library not loaded: /Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/local/lib/libRivet.dylib Referenced from: /Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so Reason: image not found make[1]: *** [analyses.html] Error 1 make: *** [all-recursive] Error 1 ls local/lib/libRivet.dylib ls: local/lib/libRivet.dylib: No such file or directory Many Thanks, Sabah. From Sabah Salih Particle Physics Group, The School of Physics and Astronomy, The University of Manchester, Schuster Laboratory, Brunswick Street, Manchester M13 9PL. Tel: +44 1612754171 or x4171 E-mail: sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk> ________________________________________ From: James William Monk [james.william.monk at cern.ch<mailto:james.william.monk at cern.ch>] Sent: 25 July 2018 13:57 To: Sabah Salih Cc: James William Monk; Andy Buckley; rivet at projects.hepforge.org<mailto:rivet at projects.hepforge.org> Subject: Re: [Rivet] ruvet and osx Hi Sabah, No, there’s no need to cross compile here, we just need to make sure it uses compilers and libs all from the same toolchain and doesn’t mix and match system tools with homebrew tools. For example, to try to use the system compiler you could try something like configuring Rivet and its dependencies with the extra config option ./configure <usual config options> CC=/usr/bin/clang CXX=/usr/bin/clang++ It looks like you prefer to use homebrew’s tools though. I don’t know where they are located on your system, but the equivalent configure command would be ./configure <usual config opts> CC=/path/to/brew/gcc CXX=/path/to/brew/g++ where you fill in the paths to the brew installation versions of gcc and g++. Of course, any Rivet dependencies (FastJet, GSL, HepMC...) will also need to be configured in the same way first. If you just want to run Rivet, as opposed to develop for it, I’d also suggest taking a look at the Docker distribution for mac: https://rivet.hepforge.org/trac/wiki/Docker cheers, James On 25 Jul 2018, at 08:37, Sabah Salih <sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>> wrote: Dear James, It is so nice to hear from you. Thank you so much for your kind reply. I just done brew tap osx-cross/arm brew tap osx-cross/avr brew install arm-gcc-bin brew install avr-gcc This look like installing gcc8 Is this what you meant? Many Tanks, Sabah. From Sabah Salih Particle Physics Group, The School of Physics and Astronomy, The University of Manchester, Schuster Laboratory, Brunswick Street, Manchester M13 9PL. Tel: +44 1612754171 or x4171 E-mail: sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk> ________________________________________ From: James William Monk [james.william.monk at cern.ch<mailto:james.william.monk at cern.ch>] Sent: 25 July 2018 01:10 To: Andy Buckley; Sabah Salih Cc: rivet at projects.hepforge.org<mailto:rivet at projects.hepforge.org> Subject: Re: [Rivet] ruvet and osx Hi, From a bit of nurdling about on various fora, that error appears to refer to an option that is present for clang, but not gcc. Clang is the supplied default on mac. You have installed a separate gcc toolchain via homebrew. What I can imagine may be happening is that autotools is detecting the system clang and adding options appropriately, but then the g++ command is now calling the gcc from homebrew. For example, on my own mac I see g++ --version Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 9.1.0 (clang-902.0.39.2) Target: x86_64-apple-darwin17.7.0 It’s possible you may be able to cure this with some appropriate environment settings like export CC=/usr/bin/clang export CXX=/usr/bin/clang++ Which would try to force it to use the system clang compiler consistently. Alternatively, try to point it to your homebrew compiler location. You may also like to consider removing homebrew as another solution - I don’t know what else it adds for you, but personally I have never found those kinds of packaging systems helpful on a mac precisely because they bring a second toolchain into play that is not always entirely consistent with the system toolchain. cheers, James On 25 Jul 2018, at 00:48, Andy Buckley <andy.buckley at cern.ch<mailto:andy.buckley at cern.ch>> wrote: I'm afraid I don't use Macs, so am not sure what is best. Using an earlier compiler might work; I'm a little surprised that autoreconf did not help. Hopefully someone else on the Rivet team has a bit more Mac experience and can help out. Andy Dr Andy Buckley, Lecturer / Royal Society University Research Fellow Particle Physics Experiment Group, University of Glasgow On Jul 24 2018, at 4:52 pm, Sabah Salih <sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>> wrote: Dear Andy, Thank you ever so much. The problem look like it is in YODA. I did cd YODA-1.7.0 autoreconf -i make that did not fix it cd .. this is where rivet-bootstrap and rerun ./rivet-bootstrap This did not fix it neither. What is the best way to get rivet to work in a mac please.? It does not have to be gcc8 Many thanks, Sabah From Sabah Salih Particle Physics Group, The School of Physics and Astronomy, The University of Manchester, Schuster Laboratory, Brunswick Street, Manchester M13 9PL. Tel: +44 1612754171 or x4171 E-mail: sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk> From: Andy Buckley [andy.buckley at cern.ch<mailto:andy.buckley at cern.ch>] Sent: 19 July 2018 14:41 To: Sabah Salih Cc: rivet at projects.hepforge.org<mailto:rivet at projects.hepforge.org> Subject: Re: [Rivet] ruvet and osx Hi Sabah, I have no idea about compilation with GCC 8 -- it looks like our "autotools" build system assumes that a compiler flag will work, but GCC 8 has removed it. I suspect that new versions of autotools would fix it, but of course our existing tarball is built with an older version. You could try building the (final) Rivet step by hand, first rebuilding the autotools bits with a call to "autoreconf -i". It may even be possible to re-run the bootstrap script after doing that, and let it finish the build with the updated Makefiles. Andy Dr Andy Buckley, Lecturer / Royal Society University Research Fellow Particle Physics Experiment Group, University of Glasgow On Jul 19 2018, at 9:15 am, Sabah Salih <sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>> wrote: Dear All, I am trying to install rivet in a max osx 10.13.6 I have gcc --version gcc (Homebrew GCC 8.1.0) 8.1.0 Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. gfortran --version GNU Fortran (Homebrew GCC 8.1.0) 8.1.0 Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. g++ --version g++ (Homebrew GCC 8.1.0) 8.1.0 Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. I got rivet from the following url wget http://rivet.hepforge.org/hg/bootstrap/raw-file/2.6.0/rivet-bootstrap chmod +x rivet-bootstrap After I run the following ./rivet-bootstrap I get the following in the end g++ -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -DNDEBUG -I/usr/include -I/usr/include -pipe -I/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/include -I/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/include -I/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/pyext/yoda -I/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/pyext/yoda -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c /Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/pyext/yoda/util.cpp -o /Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/pyext/build/temp.macosx-10.13-intel-2.7/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/pyext/yoda/util.o -O3 -Wno-unused-but-set-variable -Wno-sign-compare g++: error: unrecognized command line option '-Wshorten-64-to-32' error: command 'g++' failed with exit status 1 make[2]: *** [all-local] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all-recursive] Error 1 What do I have to do to fix it please? Maany Thanks, Sabah. From Sabah Salih Particle Physics Group, The School of Physics and Astronomy, The University of Manchester, Schuster Laboratory, Brunswick Street, Manchester M13 9PL. Tel: +44 1612754171 or x4171 E-mail: sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk> _______________________________________________ Rivet mailing list Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org> https://www.hepforge.org/lists/listinfo/rivet _______________________________________________ Rivet mailing list Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org> https://www.hepforge.org/lists/listinfo/rivet _______________________________________________ Rivet mailing list Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org> https://www.hepforge.org/lists/listinfo/rivet _______________________________________________ Rivet mailing list Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org> https://www.hepforge.org/lists/listinfo/rivet <rivet-install.txt>_______________________________________________ Rivet mailing list Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org> https://www.hepforge.org/lists/listinfo/rivet _______________________________________________ Rivet mailing list Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org> https://www.hepforge.org/lists/listinfo/rivet _______________________________________________ Rivet mailing list Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org> https://www.hepforge.org/lists/listinfo/rivet — Dr. Christian Gütschow Department of Physics and Astronomy University College London Gower Street London WC1E 6BT _______________________________________________ Rivet mailing list Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org> https://www.hepforge.org/lists/listinfo/rivet -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://www.hepforge.org/lists-archive/rivet/attachments/20180820/747f311d/attachment.html>
More information about the Rivet mailing list |