(dormant account) Reporter
	Description • 16 years ago

http://packages.debian.org/sid/dh-make
apparently dh-make is pretty automagical and that's how the mobile stuff hosted on ftp://ftp.mozilla.org/pub/mobile/dists/ is being done

	Vlad Sukhoy Assignee
	Comment 1 • 16 years ago

It looks like we want two packages here: one for plugin-enabled gcc and another for xhydra itself.

	Vlad Sukhoy Assignee
	Comment 2 • 16 years ago

This script modifies the sources of debian's gcc-4.3 package to include plugin patches from http://hg.mozilla.org/users/tglek_mozilla.com/gcc-moz-plugin-mq/ after being processed by the script, packages can be built to produce appropriate gcc binaries and libs .deb files which contain gcc with plugin support.

	Vlad Sukhoy Assignee
	Comment 3 • 16 years ago

This is a small mod to plugin.diff which adds insn headers as dependency of tree_plugin_pass. In some circumstances (such as when building pluginified debian gcc package on my machine) those headers are not available at the point when tree_plugin_pass.c is being compiled which results in build error.

For the record: this dependency is present in plugin.diff for Apple compiler.

	Vlad Sukhoy Assignee
	Comment 4 • 16 years ago

This file contains the source of debian packaging for dehydra (treehydra is also included in the package). Since there is a dependency on gcc build, the package directly uses the build files produced by the build of gcc package (which should be pluginified by above python script).

It looks like it works, I was able to run testsuite on another debian machine on the binaries installed from these packages and it passed.

I built i386 debs for gcc and dehydra: http://sukhoy.public.iastate.edu/dehydra-deb/. Not all of that is needed to get dehydra running, but the dependencies of g++-4.3 package are actually required.

In particular, here is dehydra's deb: http://sukhoy.public.iastate.edu/dehydra-deb/dehydra_0.9-1_i386.deb

Suggestions on how to handle gcc build dependency better are welcome (for now I just expect gcc package to be built in the same parent dir as dehydra).

	(dormant account) Reporter
	Comment 5 • 16 years ago

Perhaps the patched gcc src package should also build dehydra?

	Vlad Sukhoy Assignee
	Comment 6 • 16 years ago

Yeah, this makes sense.

	Vlad Sukhoy Assignee
	Comment 7 • 16 years ago

Status update: the python script and patches against debian packaging of gcc which a) apply plugin patches and b) unpack or copy dehydra and modify makefiles so that appropriate .deb file is built within gcc thus resolving this tight dependency between dehydra and gcc are located at http://hg.mozilla.org/users/vladimir.sukhoy_gmail.com/deb-gcc-plugin-dehydra-mq/

I am in the process of verifying the process on different versions of gcc packaging, it is my understanding that we can integrate this into buildbot.

One minor issue I encountered which required modifying dehydra's sources specifically for the purpose of building the package is that dehydra searches for its javascript libs in "libs" directory which is not appropriate when the .so file is located in /usr/lib. For now I just modify the source so that the path is "dehydra" and the final directory for the scripts is /usr/lib/dehydra.

	Vlad Sukhoy Assignee
	Comment 8 • 16 years ago

@Taras: This generated insn header dependency problem in plugin pass source is still relevant.. Please see the attachment for a typical debian gcc package build error due to it. Rather than doing debian-only mod, I'd like to handle that in the main patch queue, I'm pretty sure this error can be triggered with a "classic" plugin-enabled gcc build given certain combination of build flags and/or appropriate circumstances while in parallel make.

	Vlad Sukhoy Assignee
	Comment 9 • 16 years ago

Comment on attachment 325212 [details]
pluginify script wip

This code is obsolete: far superior packaging and patches evolved at above mentioned mercurial location. There's a README file there too.

	(dormant account) Reporter
	Comment 10 • 16 years ago

Comment on attachment 325213 [details] [diff] [review]
add insn headers dep for tree_plugin_pass

Seems ok

	Vlad Sukhoy Assignee
	Comment 11 • 16 years ago

Pushed insn headers thing.

	(dormant account) Reporter
	Comment 12 • 16 years ago

Hey Vlad,
I tried to follow the steps in your readme but I get this error.

	Vlad Sukhoy Assignee
	Comment 13 • 16 years ago

Something is not right: this "dehydra-unpack" patch is not a patch for dehydra but for debian packaging and should be qpush-ed onto packing's debian directory before the build and not applied onto sources during the packaging build itself which is being setup by that "pluginify_gcc.py" script. Apparently I didn't write things clearly in that README.

Here is a sequence of shell commands that should work given that the current directory is empty, writable and aptitude is setup appropriately:

apt-get source gcc-4.3 &&
hg clone http://hg.mozilla.org/users/tglek_mozilla.com/gcc-moz-plugin-mq/ &&
hg clone http://hg.mozilla.org/users/tglek_mozilla.com/dehydra-gcc &&
cd gcc-4.*/debian &&
hg init . &&
hg clone http://hg.mozilla.org/users/vladimir.sukhoy_gmail.com/deb-gcc-plugin-dehydra-mq/ .hg/patches &&
python .hg/patches/pluginify_gcc.py &&
hg qpush -a &&
cd .. &&
fakeroot debian/rules binary

Hope this helps.

	Vlad Sukhoy Assignee
	Comment 14 • 16 years ago

Status update: package repository index files generation script was added, the location at http://sukhoy.public.iastate.edu/dehydra-deb/ is now a package repository which can be listed in sources.list:

deb http://sukhoy.public.iastate.edu/dehydra-deb/ /

To assign higher priority to packages coming from the repository the following may be put to /etc/apt/preferences:

Package: *
Pin: release o=mozilla.org
Pin-Priority: 999

For now the packages from above location are i386-only. The process of generating package indices is documented in the README file.

	Vlad Sukhoy Assignee
	Comment 15 • 16 years ago

So basically here is an example buildbot config to which does the following:

1. Synchronizes debian packaging with upstream at ftp://ftp.debian.org/debian/pool/main/g/gcc-4.3/. At any given time there may be several gcc packagings there for different distributions (at the moment, there is gcc-4.3_4.3.1-9 and gcc-4.3_4.3.2-1). The build script maintains all files required to build each of these locally, downloads new files from remote and deletes older packages that are gone. If nothing is changed, the communication with upstream is only limited to downloading the file listing.

2. The patches for gcc, debian packaging and dehydra sources are updated. For each of feasible packages above, the gcc and dehydra debs are built. For now there is no support for lightweight rebuild when only dehydra changes (a lot of time can be saved by not rebuilding gcc), but I am not sure if this buildbot should be notified about dehydra changes but only ran periodically to save the effort.

The build also invokes gcc testsuite and dehydra testsuite to verify that the product is usable. The .deb files are available in the build directory for further use, such as generating indices to set up a repository. This config may be easily extended to support multiple architectures.

The versioning scheme of the result dehydra package was changed to be more friendly to debian (e.g. enable >= checks in aptitude) as follows: dehydra's version is x.y.z-gcc<gcc-ver> where x.y is the latest release (0.9) and z is the number of dehydra revisions that happened since (70), the last release tag. <gcc-ver> is the version of gcc package and it is composed of upstream version + '-plugins-' + a.b.c where a.b.c is the version of plugin patches that is generated similarly to x.y.z. So basically the full filename of a dehydra .deb file is dehydra_0.9.70-gcc4.3.1-9-plugins-0.0.24_i386.deb

This is a tentative package versioning scheme that may be appropriate, suggestions are welcome.

For now the buildbot generates no package indices because it is not clear to me what kind of indices are desired and if we want to support multiple debian distributions by providing appropriate builds or just make a simple repository as described above.

I estimate this bug as 90% complete..

	Joshua Cranmer [:jcranmer]
	Comment 16 • 11 years ago

Dehydra and treehydra are no longer maintained by Mozilla.