Task:PR1.2 autobuilder - Revision history

amigadave: {{task|completed}}

2010-05-11T08:37:59Z

← Older revision		Revision as of 08:37, 11 May 2010
Line 1:		Line 1:
		+	{{task\|completed}}
		+
	== Background ==		== Background ==

-	After the PR1.2 SDK was released, Niels deployed it to the	+	After the PR1.2 SDK was released, Niels deployed it to the autobuilder; enacting a plan which was discussed on this list. This has caused problems with libhildon dependencies, but the problems aren't just limited to that library [http://lists.maemo.org/pipermail/maemo-developers/2010-March/025668.html].
-	autobuilder; enacting a plan which was discussed on this list. This	+
-	has caused problems with libhildon dependencies, but the problems	+
-	aren't just limited to that library [http://lists.maemo.org/pipermail/maemo-developers/2010-March/025668.html].	+

	Both Niels and the Council believe we should do something to assist.		Both Niels and the Council believe we should do something to assist.
Line 11:		Line 10:
	== What we should do ==		== What we should do ==

-	We need to do something, both to improve the situation in -devel and	+	We need to do something, both to improve the situation in [[extras-devel]] and [[extras-testing]] today, and test an approach for the next upgrade.
-	-testing today, and test an approach for the next upgrade.	+

	The main requirements here are, I think:		The main requirements here are, I think:
Line 32:		Line 30:
	=== Deploy SDKs as they are released; treat -devel and -testing as trunk ===		=== Deploy SDKs as they are released; treat -devel and -testing as trunk ===

-	This is what we have now, I think we can say it doesn't work if	+	This is what we have now, I think we can say it doesn't work if there's going to be more than a few days between SDK release and device upgrade. Since Nokia doesn't pre-announce release dates, and last minute bugs could cause problems; I think we can rule this one out.
-	there's going to be more than a few days between SDK release and	+
-	device upgrade. Since Nokia doesn't pre-announce release dates, and	+
-	last minute bugs could cause problems; I think we can rule this one	+
-	out.	+

	=== Revert the builder ===		=== Revert the builder ===

-	This is basically a step backwards. "Changing the builder config is	+	This is basically a step backwards. "Changing the builder config is easy. Cleaning up -devel and rebuilding the affected packages is quite some work as we don't have any scripts for that made yet."
-	easy. Cleaning up -devel and rebuilding the affected packages is quite	+
-	some work as we don't have any scripts for that made yet."	+

	=== Hack the SDK, create some kind of hybrid ===		=== Hack the SDK, create some kind of hybrid ===

-	This'd be bad enough if limited to just libhildon, but isn't viable	+	This'd be bad enough if limited to just libhildon, but isn't viable and WILL cause unforeseen problems. Veto :-)
-	and WILL cause unforeseen problems. Veto :-)	+

	=== Create separate repos, build queues for pre- and post-1.2 ===		=== Create separate repos, build queues for pre- and post-1.2 ===

-	Similar to what's been done for Extras proper. "Creating the repos	+	Similar to what's been done for Extras proper. "Creating the repos will be about a day work, but the part that sits on top of that (management scripts, Packages interface, QA queue) will probably take a week of work."
-	will be about a day work, but the part that sits on top of that	+
-	(management scripts, Packages interface, QA queue) will probably take	+
-	a week of work."	+

	There are hard issues around QA here.		There are hard issues around QA here.
Line 60:		Line 48:
	=== Try building in each SDK in turn ===		=== Try building in each SDK in turn ===

-	My suggestion; when someone uploads to "fremantle" auto-builder it	+	My suggestion; when someone uploads to "fremantle" auto-builder it attempts to build the package against the PR1.1 SDK. If it succeeds, it goes into Extras-devel. If it fails to build, it gets built with the PR1.2 SDK, and so on.
-	attempts to build the package against the PR1.1 SDK. If it succeeds,	+
-	it goes into Extras-devel. If it fails to build, it gets built with	+
-	the PR1.2 SDK, and so on.	+

-	For an app with compile time symbol resolution (e.g. C/C++), this'd	+	For an app with compile time symbol resolution (e.g. C/C++), this'd handle both cases quite nicely. Python apps would have to depend on the specific versions of bindings which gave them the features they wanted - again, not too much of a problem.
-	handle both cases quite nicely. Python apps would have to depend on	+
-	the specific versions of bindings which gave them the features they	+
-	wanted - again, not too much of a problem.	+

-	At extras(-stable) promotion time you could even decide, based on	+	At extras(-stable) promotion time you could even decide, based on actual binary package dependencies, whether to put in fremantle-1.2 or both fremantle-1.2 & fremantle extras repos. This would fix another common complain, "what if I don't upgrade for a few weeks".
-	actual binary package dependencies, whether to put in fremantle-1.2 or	+
-	both fremantle-1.2 & fremantle extras repos. This would fix another	+
-	common complain, "what if I don't upgrade for a few weeks".	+

-	Larger packages could be prevent from being tried to built twice by	+	Larger packages could be prevent from being tried to built twice by stating a forced build dependency on a PR1.2 version of any system package.
-	stating a forced build dependency on a PR1.2 version of any system	+
-	package.	+

-	Some software won't install from -devel and -testing as it does now,	+	Some software won't install from -devel and -testing as it does now, but that will be reduced to the set of software which is actually using features that are unavailable. A HAM patch could grey out applications which were uninstallable, or show some other indication.
-	but that will be reduced to the set of software which is actually	+
-	using features that are unavailable. A HAM patch could grey out	+
-	applications which were uninstallable, or show some other indication.	+

	This is, effectively, reinventing the more intelligent [http://wiki.debian.org/Projects/ImprovedDpkgShlibdeps dpkg-shlibdeps].		This is, effectively, reinventing the more intelligent [http://wiki.debian.org/Projects/ImprovedDpkgShlibdeps dpkg-shlibdeps].

javispedro at 21:50, 16 April 2010

2010-04-16T21:50:48Z

← Older revision		Revision as of 21:50, 16 April 2010
Line 26:		Line 26:
	== Progress ==		== Progress ==

-	Fremantle autobuilder has been switched to the Squeeze devkit; idea [[#Use the improved dpkg-shlibdeps]] ~~was~~ implemented. [http://lists.maemo.org/pipermail/maemo-developers/2010-April/025870.html] [http://lists.maemo.org/pipermail/maemo-developers/2010-April/025905.html]. --~~[[User:javispedro\|javispedro]]~~ 21:50, 16 April 2010 (UTC)	+	Fremantle autobuilder has been switched to the Squeeze devkit; idea [[#Use the improved dpkg-shlibdeps]] has been implemented. [http://lists.maemo.org/pipermail/maemo-developers/2010-April/025870.html] [http://lists.maemo.org/pipermail/maemo-developers/2010-April/025905.html]. -- 21:50, 16 April 2010 (UTC)

	== Options ==		== Options ==

javispedro at 21:50, 16 April 2010

2010-04-16T21:50:25Z

← Older revision		Revision as of 21:50, 16 April 2010
Line 23:		Line 23:
	will be much appreciated.		will be much appreciated.

		+
		+	== Progress ==
		+
		+	Fremantle autobuilder has been switched to the Squeeze devkit; idea [[#Use the improved dpkg-shlibdeps]] was implemented. [http://lists.maemo.org/pipermail/maemo-developers/2010-April/025870.html] [http://lists.maemo.org/pipermail/maemo-developers/2010-April/025905.html]. --[[User:javispedro\|javispedro]] 21:50, 16 April 2010 (UTC)

	== Options ==		== Options ==

javispedro at 16:40, 11 April 2010

2010-04-11T16:40:55Z

← Older revision		Revision as of 16:40, 11 April 2010
Line 105:		Line 105:

	A nice added benefit is that we get updated debhelper in the SDK, and more Debian compatibility; might be worth to push.		A nice added benefit is that we get updated debhelper in the SDK, and more Debian compatibility; might be worth to push.
		+
		+	==== Initial testing ====
		+
		+	There will be a temporary autobuilder with the debian-squeeze devkit installed where we will experiment with .symbols files. We have thrown a few packages to it with no visible problems so far; if someone thinks his package might break when built under a newer Debian tooling please say so before it breaks.
		+
		+	.symbols file generation is currently being done externally, using the publicly released library binaries and dpkg-gensymbols. A sample PR1.2 libhildon1 .symbols file looks like [http://depot.javispedro.com/nit/syms/libhildon1.symbols this], and will be placed in /etc/dpkg/symbols/ .

	=== Case-by-case basis ===		=== Case-by-case basis ===

javispedro at 22:19, 7 April 2010

2010-04-07T22:19:36Z

← Older revision		Revision as of 22:19, 7 April 2010
Line 97:		Line 97:
	#*Fix upstream. Make them generate .symbols files, make them use the updated devkit. Probably impossible for PR1.2.		#*Fix upstream. Make them generate .symbols files, make them use the updated devkit. Probably impossible for PR1.2.
	#*Ship symbol files ourselves in an external package. A bit ugly, requires maintenance.		#*Ship symbol files ourselves in an external package. A bit ugly, requires maintenance.
		+	#Put the built packages into a common -devel repo, or specific -devel-1.0 and -devel-1.2 repositories?
		+	#If single -devel & -testing repositories, decide whether to put packages in -1.0 or -1.2 at -stable promotion time.
		+
		+	Some cons:
		+	*Just enabling debian-squeeze devkit in autobuilder might cause problems. Need to check on that. Might need testing.
		+	*Need to ensure that local SDK installations can replicate the autobuilder setup. This means debian-squeeze devkit, and .symbols files have to be distributed.

	A nice added benefit is that we get updated debhelper in the SDK, and more Debian compatibility; might be worth to push.		A nice added benefit is that we get updated debhelper in the SDK, and more Debian compatibility; might be worth to push.

javispedro: adding shlibdeps option

2010-04-07T15:14:29Z

adding shlibdeps option

← Older revision		Revision as of 15:14, 7 April 2010
Line 87:		Line 87:
	#Application uses API which is unchanged in later SDK. This will be built with PR1.1 SDK, succeeds and goes into Extras-devel. Can be promoted to Extras-testing and tested on any device with PR1.1 or PR1.2. Once promoted it COULD go into fremantle and fremantle-1.2.		#Application uses API which is unchanged in later SDK. This will be built with PR1.1 SDK, succeeds and goes into Extras-devel. Can be promoted to Extras-testing and tested on any device with PR1.1 or PR1.2. Once promoted it COULD go into fremantle and fremantle-1.2.
	#Application uses API which is introduced in later SDK. This will be built with PR1.1 SDK and fail. It will be rebuilt with PR1.2 SDK, succeed and go into Extras-devel. Can be promoted to Extras-testing and tested by testers using the most recent release. Once promoted it will go into fremantle-1.2.		#Application uses API which is introduced in later SDK. This will be built with PR1.1 SDK and fail. It will be rebuilt with PR1.2 SDK, succeed and go into Extras-devel. Can be promoted to Extras-testing and tested by testers using the most recent release. Once promoted it will go into fremantle-1.2.
		+
		+	=== Use the improved dpkg-shlibdeps ===
		+
		+	This is a big topic and as such might require more careful consideration, but seems way more future proof than the rest of options.
		+
		+	Stuff that would need to be done:
		+	#Add the debian-squeeze devkit to the autobuilder. At this point I don't know if it's usable, or not. The Lenny devkit does not contain dpkg-shlibdeps.
		+	#Ensure required .symbols files are in the SDK. We could either
		+	#*Fix upstream. Make them generate .symbols files, make them use the updated devkit. Probably impossible for PR1.2.
		+	#*Ship symbol files ourselves in an external package. A bit ugly, requires maintenance.
		+
		+	A nice added benefit is that we get updated debhelper in the SDK, and more Debian compatibility; might be worth to push.

	=== Case-by-case basis ===		=== Case-by-case basis ===

javispedro: wikifying

2010-04-07T14:34:45Z

wikifying

← Older revision		Revision as of 14:34, 7 April 2010
Line 4:		Line 4:
	autobuilder; enacting a plan which was discussed on this list. This		autobuilder; enacting a plan which was discussed on this list. This
	has caused problems with libhildon dependencies, but the problems		has caused problems with libhildon dependencies, but the problems
-	aren't just limited to that library:	+	aren't just limited to that library [http://lists.maemo.org/pipermail/maemo-developers/2010-March/025668.html].
-		+
-	http://lists.maemo.org/pipermail/maemo-developers/2010-March/025668.html	+

	Both Niels and the Council believe we should do something to assist.		Both Niels and the Council believe we should do something to assist.
Line 82:		Line 80:
	applications which were uninstallable, or show some other indication.		applications which were uninstallable, or show some other indication.

-	This is, effectively, reinventing the more intelligent ~~dpkg-shlibdeps:~~	+	This is, effectively, reinventing the more intelligent [http://wiki.debian.org/Projects/ImprovedDpkgShlibdeps dpkg-shlibdeps].
-		+
-	http://wiki.debian.org/Projects/ImprovedDpkgShlibdeps	+

	However, it deals with the following three circumstances:		However, it deals with the following three circumstances:

javispedro: New page: == Background == After the PR1.2 SDK was released, Niels deployed it to the autobuilder; enacting a plan which was discussed on this list. This has caused problems with libhildon dependen...

2010-04-07T14:32:35Z

New page: == Background == After the PR1.2 SDK was released, Niels deployed it to the autobuilder; enacting a plan which was discussed on this list. This has caused problems with libhildon dependen...

New page

== Background ==

After the PR1.2 SDK was released, Niels deployed it to the
autobuilder; enacting a plan which was discussed on this list. This
has caused problems with libhildon dependencies, but the problems
aren't just limited to that library:

http://lists.maemo.org/pipermail/maemo-developers/2010-March/025668.html

Both Niels and the Council believe we should do something to assist.

== What we should do ==

We *need* to do something, both to improve the situation in -devel and
-testing today, and test an approach for the next upgrade.

The main requirements here are, I think:

* It's not an excessive amount of work
* It's a viable long term strategy
* The QA process doesn't get broken

Thoughts and comments from developers, or anyone else with a idea,
will be much appreciated.

== Options ==

=== Deploy SDKs as they are released; treat -devel and -testing as trunk ===

This is what we have now, I think we can say it doesn't work if
there's going to be more than a few days between SDK release and
device upgrade. Since Nokia doesn't pre-announce release dates, and
last minute bugs could cause problems; I think we can rule this one
out.

=== Revert the builder ===

This is basically a step backwards. "Changing the builder config is
easy. Cleaning up -devel and rebuilding the affected packages is quite
some work as we don't have any scripts for that made yet."

=== Hack the SDK, create some kind of hybrid ===

This'd be bad enough if limited to just libhildon, but isn't viable
and WILL cause unforeseen problems. Veto :-)

=== Create separate repos, build queues for pre- and post-1.2 ===

Similar to what's been done for Extras proper. "Creating the repos
will be about a day work, but the part that sits on top of that
(management scripts, Packages interface, QA queue) will probably take
a week of work."

There are hard issues around QA here.

=== Try building in each SDK in turn ===

My suggestion; when someone uploads to "fremantle" auto-builder it
attempts to build the package against the PR1.1 SDK. If it succeeds,
it goes into Extras-devel. If it fails to build, it gets built with
the PR1.2 SDK, and so on.

For an app with compile time symbol resolution (e.g. C/C++), this'd
handle both cases quite nicely. Python apps would have to depend on
the specific versions of bindings which gave them the features they
wanted - again, not too much of a problem.

At extras(-stable) promotion time you could even decide, based on
actual binary package dependencies, whether to put in fremantle-1.2 or
both fremantle-1.2 & fremantle extras repos. This would fix another
common complain, "what if I don't upgrade for a few weeks".

Larger packages could be prevent from being tried to built twice by
stating a forced build dependency on a PR1.2 version of any system
package.

Some software won't install from -devel and -testing as it does now,
but that will be reduced to the set of software which is actually
using features that are unavailable. A HAM patch could grey out
applications which were uninstallable, or show some other indication.

This is, effectively, reinventing the more intelligent dpkg-shlibdeps:

http://wiki.debian.org/Projects/ImprovedDpkgShlibdeps

However, it deals with the following three circumstances:

#Application uses API which has changed in later SDK. This will be built with PR1.1 SDK, succeeds and goes into Extras-devel. Can be promoted to Extras-testing but there's no clear way for a tester to know it won't work if their device is running the latest OS.
#Application uses API which is unchanged in later SDK. This will be built with PR1.1 SDK, succeeds and goes into Extras-devel. Can be promoted to Extras-testing and tested on any device with PR1.1 or PR1.2. Once promoted it COULD go into fremantle and fremantle-1.2.
#Application uses API which is introduced in later SDK. This will be built with PR1.1 SDK and fail. It will be rebuilt with PR1.2 SDK, succeed and go into Extras-devel. Can be promoted to Extras-testing and tested by testers using the most recent release. Once promoted it will go into fremantle-1.2.

=== Case-by-case basis ===

Developer complains: add a temporary exception to autobuilder to build
$APPNAME with PR1.0/1.1 SDK, but goes into the shared extras-devel as
now.