[Rivet] ruvet and osx

Christian Gutschow chris.g at cern.ch
Mon Aug 20 10:55:21 BST 2018
Previous message: [Rivet] ruvet and osx
Next message: [Rivet] ruvet and osx
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hi,

sorry for having to come back to this so late. The removal of the additional ${PYTHON} is breaking the installation for Macs using Homebrew it seems (at least it did for me), since Homebrew is making its own Python installation the default. :/

Cheers,
Chris

On 30 Jul 2018, at 21:33, Andy Buckley <andy.buckley at cern.ch<mailto:andy.buckley at cern.ch>> wrote:

Is it relevant that the doc builds are explicitly called as $(PYTHON) $(srcdir)/mk-analyses-html etc., rather than just $(srcdir)/mk-analyses-html ? Maybe the implicit "shebang" execution is not such a problem... but I don't have a Mac to test on. It works fine on my Linux system without the $(PYTHON), though, so I've changed it to that and pushed the release-2-6-x head.

Really looking forward to getting past all these technical compatibility issues that crop up every time we approach this release!

Andy

Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
Particle Physics Experiment Group, University of Glasgow



On Jul 30 2018, at 8:18 pm, James William Monk <james.william.monk at cern.ch<mailto:james.william.monk at cern.ch>> wrote:

Thanks David,

How then does the rivet main executable work then? That is itself a python script that loads the Rivet libs. How does it manage to inherit the DYLD_LIBRARY_PATH

cheers,

James


On 30 Jul 2018, at 17:28, David Grellscheid <david.grellscheid at durham.ac.uk<mailto:david.grellscheid at durham.ac.uk>> wrote:

You've diagnosed it right. All programs in /usr/bin are considered protected, and will not be passed any DYLD environment. A python in /usr/local or /opt will work fine

Also, making the script executable does not help. The first line with the #! still calls either /usr/bin/env or /usr/bin/python with exactly the same issue

David (on house move teabreak)




On 30 July 2018 16:17:18 BST, James William Monk <james.william.monk at cern.ch<mailto:james.william.monk at cern.ch>> wrote:
Hi everyone,

Did we ever come across the problem that the mac System Integrity
Protection (SIP) does not pass on the DYLD_LIBRARY_PATH to a subprocess
if it thinks that subprocess is a “protected” process? I believe that
is the problem here, and is preventing the library path being set
correctly during the Rivet build because it spawns a python subprocess
that does not inherit the correct environment

It seems that if I do a python script to simply print the environment,
i.e.

import os
for key in os.environ.keys():
print key

then execute

foo=“bar” python test
(where test contains the script above)

Then I see that foo is detected in the environment. However, if I do

DYLD_LIBRARY_PATH=“bar” python test

Then DYLD_LIBRARY_PATH does not show up! Presumably SIP has decided
that the python subprocess must be protected from malicious setting of
the environment. I can see why they might think that is important,
since changing the path would potentially cause it to load a bad
version of some library. I can disable SIP to confirm this, but that
requires a reboot of my mac and is not something we should ask users to
do.

I think that the fix for this is that the mk-analysis-html script has
to be executable. That way it will inherit from the build environment,
not from the python subprocess whose environment has been wiped clean.


cheers,

James






On 30 Jul 2018, at 16:36, James William Monk
<james.william.monk at cern.ch<mailto:james.william.monk at cern.ch>> wrote:

Hi Sabah,

There is a bug in the build system for the docs in Rivet 2.6 on mac.
This issue has occurred before, but I only just remembered it. It
looks like David Grellscheid has tried to fix it in the past, but when
I use the updated Makefile.am from the doc dir, I still get the same
problem. I knew there was a reason I was still on Rivet 2.5.4 :)

It looks to me like there is a mistake in the setup for the
environment needed for the python script mk-analysis-html. At least
one error is that it seems to have the PYTHONPATH pointing to the
installation prefix, but there is nothing there of course because Rivet
hasn’t been installed yet. Another error is that DYLD_LIBRARY_PATH is
not set. I tested this by modifying doc/mk-analysis-html by adding

print "Hello from mk-analysis-html. I think $PYTHONPATH = " +
os.environ['PYTHONPATH']
print "environ keys are"
for key in os.environ.keys():
print key

This problem is only related to the building of the documentation,
but disabling doxygen doesn’t seem to help. I *think* if you plough on
regardless and do make install, you will get Rivet installed in your
prefix, just without the docs and the rivetenv.sh script.

cheers,

James

On 28 Jul 2018, at 13:11, Sabah Salih <sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>>
wrote:

Dear James,
I am very grateful for your kind help
I tried to apply rivet-bootstrap content line by line via command
line

YODA, HEPMC,... are fine
When I reached Rivet I get the following:
I attached the file which have the complete steps for RIVET

Many Thanks, Sabah.



File
"/private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/__init__.py",
line 12, in <module>
from rivet.core import *
ImportError:
dlopen(/private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so,
10): Library not loaded: /tmp/rivet/local/lib/libRivet.dylib
Referenced from:
/private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so
Reason: image not found
make[1]: *** [analyses.json] Error 1
make[1]: *** Waiting for unfinished jobs....
Using output file name 'analyses.html' and directory 'analyses'
Traceback (most recent call last):
File "mk-analysis-html", line 25, in <module>
import rivet
File
"/private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/__init__.py",
line 12, in <module>
from rivet.core import *
ImportError:
dlopen(/private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so,
10): Library not loaded: /tmp/rivet/local/lib/libRivet.dylib
Referenced from:
/private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so
Reason: image not found
make[1]: *** [analyses.html] Error 1
make: *** [all-recursive] Error 1

ls -lh
/private/tmp/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/
total 312
-rw-r--r-- 1 admin wheel 351B 3 Oct 2017 __init__.py
-rw-r--r-- 1 admin wheel 598B 28 Jul 11:26 __init__.pyc
-rw-r--r-- 1 admin wheel 3.0K 6 Nov 2017 aopaths.py
-rwxr-xr-x 1 admin wheel 121K 28 Jul 11:15 core.so
-rw-r--r-- 1 admin wheel 7.8K 3 Oct 2017 plotinfo.py
-rw-r--r-- 1 admin wheel 2.5K 3 Oct 2017 spiresbib.py
-rw-r--r-- 1 admin wheel 6.9K 20 Nov 2017 util.py


Many Thanks, Sabah.

From Sabah Salih
Particle Physics Group,
The School of Physics and Astronomy,
The University of Manchester,
Schuster Laboratory,
Brunswick Street,
Manchester M13 9PL.
Tel: +44 1612754171 or x4171
E-mail: sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>

________________________________________
From: James William Monk [james.william.monk at cern.ch<mailto:james.william.monk at cern.ch>]
Sent: 25 July 2018 22:46
To: Sabah Salih
Cc: James William Monk; Andy Buckley; rivet at projects.hepforge.org<mailto:rivet at projects.hepforge.org>
Subject: Re: [Rivet] ruvet and osx

Hi Sabah,

That error is just confirming that the Rivet lib is not present, I
presume because the build and/or installation failed. Do you see any
other error that looks like a build error? I’m not very familiar with
the bootstrap, so I can’t say whether it produces any build logs, or
where they would be, but we need to see the build failure message to
know why the libraries weren't built properly.

cheers,

James



On 25 Jul 2018, at 19:09, Sabah Salih
<sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>> wrote:

Dear James,
Thank you so much.
Following your advice I did the following:

edit rivet-bootstrap

and changed

function conf { ./configure --prefix=$INSTALL_PREFIX "$@"; }
to
function conf { ./configure CC=/usr/bin/clang CXX=/usr/bin/clang++
--prefix=$INSTALL_PREFIX "$@"; }

and
./configure --prefix=$INSTALL_PREFIX
to
./configure CC=/usr/bin/clang CXX=/usr/bin/clang++
--prefix=$INSTALL_PREFIX

When I type
./rivet-bootstrap

I get this
File
"/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/__init__.py",
line 12, in <module>
from rivet.core import *
ImportError:
dlopen(/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so,
10): Library not loaded:
/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/local/lib/libRivet.dylib
Referenced from:
/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so
Reason: image not found
make[1]: *** [analyses.json] Error 1
make[1]: *** Waiting for unfinished jobs....
Using output file name 'analyses.html' and directory 'analyses'
Traceback (most recent call last):
File "mk-analysis-html", line 25, in <module>
import rivet
File
"/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/__init__.py",
line 12, in <module>
from rivet.core import *
ImportError:
dlopen(/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so,
10): Library not loaded:
/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/local/lib/libRivet.dylib
Referenced from:
/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/Rivet-2.6.0/pyext/build/lib.macosx-10.13-intel-2.7/rivet/core.so
Reason: image not found
make[1]: *** [analyses.html] Error 1
make: *** [all-recursive] Error 1

ls local/lib/libRivet.dylib
ls: local/lib/libRivet.dylib: No such file or directory

Many Thanks, Sabah.


From Sabah Salih
Particle Physics Group,
The School of Physics and Astronomy,
The University of Manchester,
Schuster Laboratory,
Brunswick Street,
Manchester M13 9PL.
Tel: +44 1612754171 or x4171
E-mail: sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>

________________________________________
From: James William Monk [james.william.monk at cern.ch<mailto:james.william.monk at cern.ch>]
Sent: 25 July 2018 13:57
To: Sabah Salih
Cc: James William Monk; Andy Buckley; rivet at projects.hepforge.org<mailto:rivet at projects.hepforge.org>
Subject: Re: [Rivet] ruvet and osx

Hi Sabah,

No, there’s no need to cross compile here, we just need to make
sure it uses compilers and libs all from the same toolchain and doesn’t
mix and match system tools with homebrew tools. For example, to try to
use the system compiler you could try something like configuring Rivet
and its dependencies with the extra config option

./configure <usual config options> CC=/usr/bin/clang
CXX=/usr/bin/clang++

It looks like you prefer to use homebrew’s tools though. I don’t
know where they are located on your system, but the equivalent
configure command would be

./configure <usual config opts> CC=/path/to/brew/gcc
CXX=/path/to/brew/g++

where you fill in the paths to the brew installation versions of
gcc and g++. Of course, any Rivet dependencies (FastJet, GSL,
HepMC...) will also need to be configured in the same way first.

If you just want to run Rivet, as opposed to develop for it, I’d
also suggest taking a look at the Docker distribution for mac:

https://rivet.hepforge.org/trac/wiki/Docker

cheers,

James


On 25 Jul 2018, at 08:37, Sabah Salih
<sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>> wrote:

Dear James,
It is so nice to hear from you.
Thank you so much for your kind reply.

I just done

brew tap osx-cross/arm
brew tap osx-cross/avr

brew install arm-gcc-bin

brew install avr-gcc
This look like installing gcc8

Is this what you meant?

Many Tanks, Sabah.

From Sabah Salih
Particle Physics Group,
The School of Physics and Astronomy,
The University of Manchester,
Schuster Laboratory,
Brunswick Street,
Manchester M13 9PL.
Tel: +44 1612754171 or x4171
E-mail: sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>

________________________________________
From: James William Monk [james.william.monk at cern.ch<mailto:james.william.monk at cern.ch>]
Sent: 25 July 2018 01:10
To: Andy Buckley; Sabah Salih
Cc: rivet at projects.hepforge.org<mailto:rivet at projects.hepforge.org>
Subject: Re: [Rivet] ruvet and osx

Hi,

From a bit of nurdling about on various fora, that error appears
to refer to an option that is present for clang, but not gcc. Clang is
the supplied default on mac. You have installed a separate gcc
toolchain via homebrew. What I can imagine may be happening is that
autotools is detecting the system clang and adding options
appropriately, but then the g++ command is now calling the gcc from
homebrew. For example, on my own mac I see

g++ --version
Configured with:
--prefix=/Applications/Xcode.app/Contents/Developer/usr
--with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 9.1.0 (clang-902.0.39.2)
Target: x86_64-apple-darwin17.7.0

It’s possible you may be able to cure this with some appropriate
environment settings like

export CC=/usr/bin/clang
export CXX=/usr/bin/clang++

Which would try to force it to use the system clang compiler
consistently. Alternatively, try to point it to your homebrew compiler
location. You may also like to consider removing homebrew as another
solution - I don’t know what else it adds for you, but personally I
have never found those kinds of packaging systems helpful on a mac
precisely because they bring a second toolchain into play that is not
always entirely consistent with the system toolchain.

cheers,

James



On 25 Jul 2018, at 00:48, Andy Buckley <andy.buckley at cern.ch<mailto:andy.buckley at cern.ch>>
wrote:

I'm afraid I don't use Macs, so am not sure what is best. Using
an earlier compiler might work; I'm a little surprised that autoreconf
did not help. Hopefully someone else on the Rivet team has a bit more
Mac experience and can help out.

Andy

Dr Andy Buckley, Lecturer / Royal Society University Research
Fellow
Particle Physics Experiment Group, University of Glasgow
On Jul 24 2018, at 4:52 pm, Sabah Salih
<sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>> wrote:


Dear Andy,
Thank you ever so much. The problem look like it is in
YODA.
I did

cd YODA-1.7.0

autoreconf -i
make

that did not fix it

cd ..
this is where rivet-bootstrap

and rerun

./rivet-bootstrap

This did not fix it neither.

What is the best way to get rivet to work in a mac please.?
It does not have to be gcc8

Many thanks, Sabah


From Sabah Salih
Particle Physics Group,
The School of Physics and Astronomy,
The University of Manchester,
Schuster Laboratory,
Brunswick Street,
Manchester M13 9PL.
Tel: +44 1612754171 or x4171
E-mail: sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>
From: Andy Buckley [andy.buckley at cern.ch<mailto:andy.buckley at cern.ch>]
Sent: 19 July 2018 14:41
To: Sabah Salih
Cc: rivet at projects.hepforge.org<mailto:rivet at projects.hepforge.org>
Subject: Re: [Rivet] ruvet and osx

Hi Sabah,
I have no idea about compilation with GCC 8 -- it looks like our
"autotools" build system assumes that a compiler flag will work, but
GCC 8 has removed it. I suspect that new versions of autotools would
fix it, but of course our existing tarball is built with an older
version.

You could try building the (final) Rivet step by hand, first
rebuilding the autotools bits with a call to "autoreconf -i". It may
even be possible to re-run the bootstrap script after doing that, and
let it finish the build with the updated Makefiles.

Andy

Dr Andy Buckley, Lecturer / Royal Society University Research
Fellow
Particle Physics Experiment Group, University of Glasgow
On Jul 19 2018, at 9:15 am, Sabah Salih
<sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>> wrote:


Dear All,
I am trying to install rivet in a max osx 10.13.6
I have

gcc --version
gcc (Homebrew GCC 8.1.0) 8.1.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

gfortran --version
GNU Fortran (Homebrew GCC 8.1.0) 8.1.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

g++ --version
g++ (Homebrew GCC 8.1.0) 8.1.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.


I got rivet from the following url

wget
http://rivet.hepforge.org/hg/bootstrap/raw-file/2.6.0/rivet-bootstrap
chmod +x rivet-bootstrap

After I run the following
./rivet-bootstrap

I get the following in the end


g++ -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe
-fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX
-DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g
-fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -DNDEBUG
-I/usr/include -I/usr/include -pipe
-I/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/include
-I/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/include
-I/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/pyext/yoda
-I/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/pyext/yoda
-I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
-c
/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/pyext/yoda/util.cpp
-o
/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/pyext/build/temp.macosx-10.13-intel-2.7/Users/admin/madgraph/MG5_aMC_v2_6_3/rivet/YODA-1.7.0/pyext/yoda/util.o
-O3 -Wno-unused-but-set-variable -Wno-sign-compare
g++: error: unrecognized command line option '-Wshorten-64-to-32'
error: command 'g++' failed with exit status 1
make[2]: *** [all-local] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive] Error 1


What do I have to do to fix it please?


Maany Thanks, Sabah.

From Sabah Salih
Particle Physics Group,
The School of Physics and Astronomy,
The University of Manchester,
Schuster Laboratory,
Brunswick Street,
Manchester M13 9PL.
Tel: +44 1612754171 or x4171
E-mail: sabah.salih at manchester.ac.uk<mailto:sabah.salih at manchester.ac.uk>
_______________________________________________
Rivet mailing list
Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
https://www.hepforge.org/lists/listinfo/rivet
_______________________________________________
Rivet mailing list
Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
https://www.hepforge.org/lists/listinfo/rivet

_______________________________________________
Rivet mailing list
Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
https://www.hepforge.org/lists/listinfo/rivet

_______________________________________________
Rivet mailing list
Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
https://www.hepforge.org/lists/listinfo/rivet

<rivet-install.txt>_______________________________________________
Rivet mailing list
Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
https://www.hepforge.org/lists/listinfo/rivet

_______________________________________________
Rivet mailing list
Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
https://www.hepforge.org/lists/listinfo/rivet
_______________________________________________
Rivet mailing list
Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
https://www.hepforge.org/lists/listinfo/rivet






 —

 Dr. Christian Gütschow

 Department of Physics and Astronomy
 University College London
 Gower Street
 London WC1E 6BT



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.hepforge.org/lists-archive/rivet/attachments/20180820/ffa3d726/attachment.html>
Previous message: [Rivet] ruvet and osx
Next message: [Rivet] ruvet and osx
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the Rivet mailing list