Gentoo! What did you say?

Finding CPU flags using gcc

2011-07-27T18:13:00.000+03:00

Hi all,
One of Gentoo's most attractive features is the ability to specify the cpu CFLAGS, in order to compile all the system to use as much of the hardware as possible.
This leads us to finding the correct cpu flags to include in our /etc/make.conf file.
The approach I was using until now, is to find my cpu flags in the Gentoo wiki page: Safe Cflags. There you can find your cpu by using 'cat /proc/cpuinfo'.
Another approach is to use the -march=native flag, which tells gcc to find the correct CFLAGS for my cpu.
When you compile systems for embedded or other targets, you can't use -march=native, because all the packages will be built for the running cpu and not the target cpu. To compile for the target cpu, you will need to find the flags that suits your target cpu, then apply them in your build environment.

Today I encountered a post from Andy Wilkinson on the Gentoo user mailing list. He describes a very neat trick to find our cpu CFLAGS.

gcc -### -e -v -march=native /usr/include/stdlib.h

This command output will show COLLECT_GCC_OPTIONS which are the options gcc find about your cpu when using -march=native. Here is the output:


COLLECT_GCC_OPTIONS='-e' '-v'  '-fPIE' '-pie'
 "/usr/libexec/gcc/i686-pc-linux-gnu/4.4.5/cc1" "-quiet" "/usr/include/stdlib.h" "-D_FORTIFY_SOURCE=2" "-march=core2" "-mcx16" "-msahf" "-msse4.1" "--param" "l1-cache-size=32" "--param" "l1-cache-line-size=64" "--param" "l2-cache-size=3072" "-mtune=core2" "-fno-strict-overflow" "-quiet" "-dumpbase" "stdlib.h" "-auxbase" "stdlib" "-fPIE" "-o" "/tmp/cc8bTF68.s" "--output-pch=/usr/include/stdlib.h.gch"

As you see, my cpu is core2 with msse4.1. You also get all the params of the cache size for free.

Regards,
Kfir

What the hell did I install ?!

2010-10-03T17:53:00.000+02:00

So I got a new laptop from my work. I have already my IBM x60s running Gentoo. I want to install my new Lenovo x200 with about the same configuration.
So what did I installed on my IBM x60s?

cat /var/lib/portage/world

shows you all the packages you have ran 'emerge package'. It will not show you all the dependencies that portage have added, so its a clean list of packages you installed.

I copy the world file to my fresh install and edit it deleting packages I don't need.
But wait! Don't install this packages yet. You will download lot of blobs that you have already.
In my x60s all the packages are stored in /usr/portage/distfiles/.
So I just copy all the files from the old computer to the new computer. I do it with rsync, but you can do it with scp, cp+sshfs etc.

rsync -avPn --exclude=svn-src /usr/portage/distfiles/* x200/usr/portage/distfiles/

Lets make sure we have all the blobs:

emerge --fetchonly $(cat copied.world.file)

Now we can install everything ;-)

emerge $(cat copied.world.file)

Simple!

Regards,
Kfir

What does this USE flag do?

2010-04-28T13:26:00.000+03:00

Hi,
Today I ran world update and saw that the kernel sources have a new USE flag: deblob.
What does deblob do?
There are 3 ways that I know of to find the answer.
Here they are:
The first is using the command line 'euse' from app-portage/gentoolkit:

 $ euse -I deblob
global use flags (searching: deblob)
************************************************************
no matching entries found

local use flags (searching: deblob)
************************************************************
[+ C  ] deblob (sys-kernel/gentoo-sources):
Remove binary blobs from kernel sources to provide libre license compliance.

Much faster option is using 'quse' from app-portage/portage-utils:

 $ quse -D deblob
 local:deblob:sys-kernel/ck-sources: Remove binary blobs from kernel sources to provide libre license compliance.
 local:deblob:sys-kernel/gentoo-sources: Remove binary blobs from kernel sources to provide libre license compliance.
 local:deblob:sys-kernel/git-sources: Remove binary blobs from kernel sources to provide libre license compliance.
 local:deblob:sys-kernel/hardened-sources: Remove binary blobs from kernel sources to provide libre license compliance.
 local:deblob:sys-kernel/mips-sources: Remove binary blobs from kernel sources to provide libre license compliance.
 local:deblob:sys-kernel/mm-sources: Remove binary blobs from kernel sources to provide libre license compliance.
 local:deblob:sys-kernel/openvz-sources: Remove binary blobs from kernel sources to provide libre license compliance.
 local:deblob:sys-kernel/tuxonice-sources: Remove binary blobs from kernel sources to provide libre license compliance.
 local:deblob:sys-kernel/vanilla-sources: Remove binary blobs from kernel sources to provide libre license compliance.
 local:deblob:sys-kernel/vserver-sources: Remove binary blobs from kernel sources to provide libre license compliance.
 local:deblob:sys-kernel/xen-sources: Remove binary blobs from kernel sources to provide libre license compliance.
 local:deblob:sys-kernel/zen-sources: Remove binary blobs from kernel sources to provide libre license compliance.

And finally the option from the Gentoo handbook USE flags:

$ cat /usr/portage/profiles/use.desc | grep deblob

The last option doesn't always produce results. Its better to use the first two (euse, quse).

Regards,
Kfir

Searching for installed packages in special section/category

2010-04-16T20:18:00.002+03:00

$ for i in /usr/portage/dev-ruby/*; do eix -I --compact dev-ruby/$(basename $i); done  | grep I

This will show us all the installed packages under /usr/portage/dev-ruby.
How can we do it better?

$ equery list -i | grep dev-ruby

So we need to find a way to tell equery or eix to search just in dev-ruby category.
I asked this question in Gentoo Forums and got the answer I was looking for. Hehe.
Its:

eix -IcA dev-ruby

-I is for installed packages.
-c is for compact output (one line per package)
-A is the most important flag, we searched for and it is the --category-name
So we came back to eix and it solved our problems.
Sometimes you read the manual page, and you just don't find what you are looking for, even though its there obvious in plain text ;-)

In the thread where I asked the question, I got a few more ways to do this.

$  ls /var/db/pkg/dev-ruby/
cmdparse-2.0.2  rubygems-1.3.5

Another way is using qlist from app-portage/portage-utils this way:

$  qlist -I dev-ruby
dev-ruby/cmdparse
dev-ruby/rubygems

This qlist command is equivalent to this eix command:

$ eix -I --only-names dev-ruby/
dev-ruby/cmdparse
dev-ruby/rubygems

In the last command the / after dev-ruby is important. qlist doesn't have this problem.

I didn't go deeply how every command is working, but as a solution to our discussion, this is more then enough ;-)

Kfir

What package contain this file?

2010-04-16T13:28:00.010+03:00

Hi,
We have a file we want to use, but we don't know which package contain this file. How can we find the package containing this file? Read on....

Lets install equery.
So lets search and see if there is a package named equery:

$ eix equery

Well yes,

No matches found

So let us try search the description of packages for equery:

$ eix -S equery
[I] app-shells/gentoo-bashcomp
     Available versions:  20090613 ~20091225
     Installed versions:  20090613(14:24:23 01/04/10)
     Homepage:            http://www.gentoo.org/
     Description:         Gentoo-specific bash command-line completions (emerge, ebuild, equery, repoman, layman, etc)

Bash completion is not what we need.
So we left with using equery to search for equery ;-)
equery utility is installed in my computer so lets use it.
Let's run equery alone:

$ equery       
!!! No command or unknown command given
Usage: equery <global-opts> command <local-opts>
where <global-opts> is one of
 -q, --quiet   - minimal output
 -C, --nocolor - turn off colours
 -h, --help    - this help screen
 -V, --version - display version info
 -N, --no-pipe - turn off pipe detection
where command(short) is one of
 belongs(b) <local-opts> files... - list all packages owning files...
 changes(c)  - not implemented yet
 check(k) pkgspec - check MD5sums and timestamps of pkgspec's files
 depends(d) <local-opts> pkgspec - list all direct dependencies matching pkgspec
 depgraph(g) <local-opts> pkgspec - display a dependency tree for pkgspec
 files(f) <local-opts> pkgspec - list files owned by pkgspec
 glsa(a)  - not implemented yet
 hasuse(h) <local-opts> useflag - list all packages with useflag
 list(l) <local-opts> pkgspec - list all packages matching pkgspec
 size(s) <local-opts> pkgspec - print size of files contained in package pkgspec
 stats(t)  - not implemented yet
 uses(u) <local-opts> pkgspec - display USE flags for pkgspec
 which(w) pkgspec - print full path to ebuild for package pkgspec

The line:

belongs(b) <local-opts> files... - list all packages owning files...

Is what we want to use to search what package installs the utility equery. Perfect!
So we run:

$ equery belongs $(which equery)
[ Searching for file(s) /usr/bin/equery in *... ]
app-portage/gentoolkit-0.2.4.6.1-r1 (/usr/bin/equery)

Oh, ok, now we know that /usr/bin/equery is installed by app-portage/gentoolkit.
Now to use it, just

$ emerge -av app-portage/gentoolkit

and enjoy :-)

Kfir

Multiple portage SYNC servers

2010-04-16T01:53:00.009+03:00

My home server died, and it will be down until I figure out what is the problem.
The server is syncing portage everyday and share the portage tree in the local network. This lets me sync my laptop against my local server. Faster and no double rsync on remote rsync servers.
This setup is pretty simple. You can learn how to do it here.
The important thing is to define the SYNC variable in make.conf as:

SYNC="rsync://LOCAL_SERVER_IP/gentoo-portage"

After my server died, I just added a real server to SYNC line:

SYNC="rsync://LOCAL_SERVER_IP/gentoo-portage rsync://linux.rz.ruhr-uni-bochum.de/gentoo-portage/"

Exactly as we use GENTOO_MIRRORS variable.
Nope, it doesn't work. I have asked in Gentoo forums here why there is no multiple sync servers supported.
I have solved it like this in a standalone script:

SYNC_SERVERS="rsync://LOCALSERVER/gentoo-portage rsync://linux.rz.ruhr-uni-bochum.de/gentoo-portage/ rsync://mirror.isoc.org.il/gentoo-portage/"
for server in $SYNC_SERVERS
do
        SYNC="$server" eix-sync -q && break
done

This behaviour of SYNC variable in make.conf should be corected to support multiple rsync servers.

Regards,
Kfir