Welcome to Soft32 Linux Forums!
FAQFAQ    SearchSearch      ProfileProfile    Private MessagesPrivate Messages   Log inLog in

[RFC] new -stable tag variant, Git workflow question

 
Goto page 1, 2, 3
   Soft32 Home -> Linux -> Kernel RSS
Next:  Bug#473205: ITA: valknut -- Qt 3 client for Direc..  
Author Message
Ingo Molnar

External


Since: Nov 05, 2003
Posts: 2921



(Msg. 1) Posted: Mon Nov 09, 2009 11:20 pm
Post subject: [RFC] new -stable tag variant, Git workflow question
Archived from groups: linux>kernel (more info?)

FYI, today i committed a scheduler performance fix that has a number of
commit prerequisites for -stable integration. Those commits are not
marked -stable.

Previously, in similar situations, i solved it by email-forwarding the
prereq commits to stable RemoveThis @kernel.org. (or by waiting for your 'it does
not apply to -stable' email and figuring out the prereqs then.)

But we can move this into the Git commit space too, and minimize the
work for the -stable team, via a new -stable tag variant. So with this
new commit i started using a new tagging scheme in the commit itself:

Cc: <stable RemoveThis @kernel.org> # .32.x: a1f84a3: sched: Check for an idle shared cache
Cc: <stable RemoveThis @kernel.org> # .32.x: 1b9508f: sched: Rate-limit newidle
Cc: <stable RemoveThis @kernel.org> # .32.x: fd21073: sched: Fix affinity logic
Cc: <stable RemoveThis @kernel.org> # .32.x
LKML-Reference: <1257821402.5648.17.camel RemoveThis @marge.simson.net>
Signed-off-by: Ingo Molnar <mingo RemoveThis @elte.hu>

The tag sequence has the meaning of:

git cherry-pick a1f84a3
git cherry-pick 1b9508f
git cherry-pick fd21073
git cherry-pick <this commit>

and i'm wondering whether this tagging scheme is fine with your -stable
scripting, etc.

A further question is, i can see using this tagging scheme in the future
in merge commits log messages too - will your scripts notice that too?

For example if there's a few commits left in tip:*/urgent branches
(tip:sched/urgent, tip:core/urgent, tip:x86/urgent, etc.) by the time
v2.6.32 is released, i will then merge them into tip:sched/core,
tip:core/core, tip:x86/core, etc. - and i could use the merge commit as
a notification area to 'activate' them for -stable backporting via a
merge commit.

This is how such merge commits would look like:

Merge branch 'core/urgent' into core/rcu

Merge reason: Pick up urgent fixes that did not make it into .32.0

Cc: <stable RemoveThis @kernel.org> # .32.x: 83f5b01: rcu: Fix long-grace-period race
Signed-off-by: Ingo Molnar <mingo RemoveThis @elte.hu>

This is not so rare of a situation as it might seem - for the trees i
maintain it happens in almost every release cycle - i typically skip
urgent branch merges after -rc8/-rc9, unless they are very-very-urgent
fixes - but they'd still be eligible for -stable.

I've attached the full commit below. The prereq commits are not uptream
yet, and they dont carry a -stable backporting tag as the -stable
relevance was not anticipated at that point yet. They will all be
upstream in the next merge window when Linus merges the relevant tree -
and then all these tags become visible to the -stable team's scripts.

What do you think about this new -stable tagging variant? To me it looks
quite intuitive, less error-prone and it is more informative as well.
Furthermore, it gives us some freedom to mark commits as backport
candidates later on. I kept them oneliners for the purpose of making
them all self-sufficient tags.

( Sidenote: i wouldnt go as far as to generate null Git commits to mark
backports after the fact - this scheme is for a series of commits that
get 'completed' - there's usually a final followup commit that can
embedd this information. )

Ingo

---------------------------->
From eae0c9dfb534cb3449888b9601228efa6480fdb5 Mon Sep 17 00:00:00 2001
From: Mike Galbraith <efault RemoveThis @gmx.de>
Date: Tue, 10 Nov 2009 03:50:02 +0100
Subject: [PATCH] sched: Fix and clean up rate-limit newidle code

Commit 1b9508f, "Rate-limit newidle" has been confirmed to fix
the netperf UDP loopback regression reported by Alex Shi.

This is a cleanup and a fix:

- moved to a more out of the way spot

- fix to ensure that balancing doesn't try to balance
runqueues which haven't gone online yet, which can
mess up CPU enumeration during boot.

Reported-by: Alex Shi <alex.shi RemoveThis @intel.com>
Reported-by: Zhang, Yanmin <yanmin_zhang RemoveThis @linux.intel.com>
Signed-off-by: Mike Galbraith <efault RemoveThis @gmx.de>
Acked-by: Peter Zijlstra <a.p.zijlstra RemoveThis @chello.nl>
Cc: <stable RemoveThis @kernel.org> # .32.x: a1f84a3: sched: Check for an idle shared cache
Cc: <stable RemoveThis @kernel.org> # .32.x: 1b9508f: sched: Rate-limit newidle
Cc: <stable RemoveThis @kernel.org> # .32.x: fd21073: sched: Fix affinity logic
Cc: <stable RemoveThis @kernel.org> # .32.x
LKML-Reference: <1257821402.5648.17.camel RemoveThis @marge.simson.net>
Signed-off-by: Ingo Molnar <mingo RemoveThis @elte.hu>
---
kernel/sched.c | 28 +++++++++++++++-------------
1 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/kernel/sched.c b/kernel/sched.c
index 23e3535..ad37776 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -2354,17 +2354,6 @@ static int try_to_wake_up(struct task_struct *p, unsigned int state,
if (rq != orig_rq)
update_rq_clock(rq);

- if (rq->idle_stamp) {
- u64 delta = rq->clock - rq->idle_stamp;
- u64 max = 2*sysctl_sched_migration_cost;
-
- if (delta > max)
- rq->avg_idle = max;
- else
- update_avg(&rq->avg_idle, delta);
- rq->idle_stamp = 0;
- }
-
WARN_ON(p->state != TASK_WAKING);
cpu = task_cpu(p);

@@ -2421,6 +2410,17 @@ out_running:
#ifdef CONFIG_SMP
if (p->sched_class->task_wake_up)
p->sched_class->task_wake_up(rq, p);
+
+ if (unlikely(rq->idle_stamp)) {
+ u64 delta = rq->clock - rq->idle_stamp;
+ u64 max = 2*sysctl_sched_migration_cost;
+
+ if (delta > max)
+ rq->avg_idle = max;
+ else
+ update_avg(&rq->avg_idle, delta);
+ rq->idle_stamp = 0;
+ }
#endif
out:
task_rq_unlock(rq, &flags);
@@ -4098,7 +4098,7 @@ static int load_balance(int this_cpu, struct rq *this_rq,
unsigned long flags;
struct cpumask *cpus = __get_cpu_var(load_balance_tmpmask);

- cpumask_setall(cpus);
+ cpumask_copy(cpus, cpu_online_mask);

/*
* When power savings policy is enabled for the parent domain, idle
@@ -4261,7 +4261,7 @@ load_balance_newidle(int this_cpu, struct rq *this_rq, struct sched_domain *sd)
int all_pinned = 0;
struct cpumask *cpus = __get_cpu_var(load_balance_tmpmask);

- cpumask_setall(cpus);
+ cpumask_copy(cpus, cpu_online_mask);

/*
* When power savings policy is enabled for the parent domain, idle
@@ -9522,6 +9522,8 @@ void __init sched_init(void)
rq->cpu = i;
rq->online = 0;
rq->migration_thread = NULL;
+ rq->idle_stamp = 0;
+ rq->avg_idle = 2*sysctl_sched_migration_cost;
INIT_LIST_HEAD(&rq->migration_queue);
rq_attach_root(rq, &def_root_domain);
#endif

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo RemoveThis @vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
Greg KH

External


Since: Nov 03, 2006
Posts: 1124



(Msg. 2) Posted: Mon Nov 09, 2009 11:20 pm
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On Tue, Nov 10, 2009 at 04:48:31AM +0100, Ingo Molnar wrote:
> FYI, today i committed a scheduler performance fix that has a number of
> commit prerequisites for -stable integration. Those commits are not
> marked -stable.
>
> Previously, in similar situations, i solved it by email-forwarding the
> prereq commits to stable.TakeThisOut@kernel.org. (or by waiting for your 'it does
> not apply to -stable' email and figuring out the prereqs then.)
>
> But we can move this into the Git commit space too, and minimize the
> work for the -stable team, via a new -stable tag variant. So with this
> new commit i started using a new tagging scheme in the commit itself:
>
> Cc: <stable.TakeThisOut@kernel.org> # .32.x: a1f84a3: sched: Check for an idle shared cache
> Cc: <stable.TakeThisOut@kernel.org> # .32.x: 1b9508f: sched: Rate-limit newidle
> Cc: <stable.TakeThisOut@kernel.org> # .32.x: fd21073: sched: Fix affinity logic
> Cc: <stable.TakeThisOut@kernel.org> # .32.x
> LKML-Reference: <1257821402.5648.17.camel.TakeThisOut@marge.simson.net>
> Signed-off-by: Ingo Molnar <mingo.TakeThisOut@elte.hu>
>
> The tag sequence has the meaning of:
>
> git cherry-pick a1f84a3
> git cherry-pick 1b9508f
> git cherry-pick fd21073
> git cherry-pick <this commit>
>
> and i'm wondering whether this tagging scheme is fine with your -stable
> scripting, etc.

It would work just fine.

I only rely on one main script right now, one that runs from James's
directory on kernel.org that picks out the "Cc: <stable.TakeThisOut@kernel.org>"
lines and forwards the full commit message to stable.TakeThisOut@kernel.org.

Then I have a simple script that I just pass the git commit id and
formats it properly for inclusion on the quilt tree for the stable
queue. If you list the other git commit ids that are needed as a
prerequesite as you did above, that's trivial to also pick out.

So I think this is good for me and my workflow.

> A further question is, i can see using this tagging scheme in the future
> in merge commits log messages too - will your scripts notice that too?

Hm, I don't think we look at merges as there's nothing there to actually
commit.

> For example if there's a few commits left in tip:*/urgent branches
> (tip:sched/urgent, tip:core/urgent, tip:x86/urgent, etc.) by the time
> v2.6.32 is released, i will then merge them into tip:sched/core,
> tip:core/core, tip:x86/core, etc. - and i could use the merge commit as
> a notification area to 'activate' them for -stable backporting via a
> merge commit.
>
> This is how such merge commits would look like:
>
> Merge branch 'core/urgent' into core/rcu
>
> Merge reason: Pick up urgent fixes that did not make it into .32.0
>
> Cc: <stable.TakeThisOut@kernel.org> # .32.x: 83f5b01: rcu: Fix long-grace-period race
> Signed-off-by: Ingo Molnar <mingo.TakeThisOut@elte.hu>
>
> This is not so rare of a situation as it might seem - for the trees i
> maintain it happens in almost every release cycle - i typically skip
> urgent branch merges after -rc8/-rc9, unless they are very-very-urgent
> fixes - but they'd still be eligible for -stable.

Ok, that would be good and fine with me.

James, would your script pick this up, or does it need to also pay
attention to merge commits?

> I've attached the full commit below. The prereq commits are not uptream
> yet, and they dont carry a -stable backporting tag as the -stable
> relevance was not anticipated at that point yet. They will all be
> upstream in the next merge window when Linus merges the relevant tree -
> and then all these tags become visible to the -stable team's scripts.
>
> What do you think about this new -stable tagging variant? To me it looks
> quite intuitive, less error-prone and it is more informative as well.
> Furthermore, it gives us some freedom to mark commits as backport
> candidates later on. I kept them oneliners for the purpose of making
> them all self-sufficient tags.

I agree.

> ( Sidenote: i wouldnt go as far as to generate null Git commits to mark
> backports after the fact - this scheme is for a series of commits that
> get 'completed' - there's usually a final followup commit that can
> embedd this information. )

That's fine, don't worry about this.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo.TakeThisOut@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
Ingo Molnar

External


Since: Nov 05, 2003
Posts: 2921



(Msg. 3) Posted: Mon Nov 09, 2009 11:20 pm
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

* Greg KH <gregkh RemoveThis @suse.de> wrote:

> > I've attached the full commit below. The prereq commits are not
> > uptream yet, and they dont carry a -stable backporting tag as the
> > -stable relevance was not anticipated at that point yet. They will
> > all be upstream in the next merge window when Linus merges the
> > relevant tree - and then all these tags become visible to the
> > -stable team's scripts.
> >
> > What do you think about this new -stable tagging variant? To me it
> > looks quite intuitive, less error-prone and it is more informative
> > as well. Furthermore, it gives us some freedom to mark commits as
> > backport candidates later on. I kept them oneliners for the purpose
> > of making them all self-sufficient tags.
>
> I agree.

Ok - thanks for the confirmation - i've pushed out the first such
commit. (Let me know if there's any problem with it down the line - it
will be a few weeks, in the next merge window, until it truly
'activates' for -stable.)

Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo RemoveThis @vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
James Bottomley

External


Since: Sep 10, 2003
Posts: 82



(Msg. 4) Posted: Tue Nov 10, 2009 9:20 am
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On Mon, 2009-11-09 at 20:14 -0800, Greg KH wrote:
> On Tue, Nov 10, 2009 at 04:48:31AM +0100, Ingo Molnar wrote:
> > FYI, today i committed a scheduler performance fix that has a number of
> > commit prerequisites for -stable integration. Those commits are not
> > marked -stable.
> >
> > Previously, in similar situations, i solved it by email-forwarding the
> > prereq commits to stable.DeleteThis@kernel.org. (or by waiting for your 'it does
> > not apply to -stable' email and figuring out the prereqs then.)
> >
> > But we can move this into the Git commit space too, and minimize the
> > work for the -stable team, via a new -stable tag variant. So with this
> > new commit i started using a new tagging scheme in the commit itself:
> >
> > Cc: <stable.DeleteThis@kernel.org> # .32.x: a1f84a3: sched: Check for an idle shared cache
> > Cc: <stable.DeleteThis@kernel.org> # .32.x: 1b9508f: sched: Rate-limit newidle
> > Cc: <stable.DeleteThis@kernel.org> # .32.x: fd21073: sched: Fix affinity logic
> > Cc: <stable.DeleteThis@kernel.org> # .32.x
> > LKML-Reference: <1257821402.5648.17.camel.DeleteThis@marge.simson.net>
> > Signed-off-by: Ingo Molnar <mingo.DeleteThis@elte.hu>
> >
> > The tag sequence has the meaning of:
> >
> > git cherry-pick a1f84a3
> > git cherry-pick 1b9508f
> > git cherry-pick fd21073
> > git cherry-pick <this commit>
> >
> > and i'm wondering whether this tagging scheme is fine with your -stable
> > scripting, etc.
>
> It would work just fine.
>
> I only rely on one main script right now, one that runs from James's
> directory on kernel.org that picks out the "Cc: <stable.DeleteThis@kernel.org>"
> lines and forwards the full commit message to stable.DeleteThis@kernel.org.
>
> Then I have a simple script that I just pass the git commit id and
> formats it properly for inclusion on the quilt tree for the stable
> queue. If you list the other git commit ids that are needed as a
> prerequesite as you did above, that's trivial to also pick out.
>
> So I think this is good for me and my workflow.

So I take this to mean that I don't alter my script and you pick out the
precursors with yours ...

> > A further question is, i can see using this tagging scheme in the future
> > in merge commits log messages too - will your scripts notice that too?
>
> Hm, I don't think we look at merges as there's nothing there to actually
> commit.
>
> > For example if there's a few commits left in tip:*/urgent branches
> > (tip:sched/urgent, tip:core/urgent, tip:x86/urgent, etc.) by the time
> > v2.6.32 is released, i will then merge them into tip:sched/core,
> > tip:core/core, tip:x86/core, etc. - and i could use the merge commit as
> > a notification area to 'activate' them for -stable backporting via a
> > merge commit.
> >
> > This is how such merge commits would look like:
> >
> > Merge branch 'core/urgent' into core/rcu
> >
> > Merge reason: Pick up urgent fixes that did not make it into .32.0
> >
> > Cc: <stable.DeleteThis@kernel.org> # .32.x: 83f5b01: rcu: Fix long-grace-period race
> > Signed-off-by: Ingo Molnar <mingo.DeleteThis@elte.hu>
> >
> > This is not so rare of a situation as it might seem - for the trees i
> > maintain it happens in almost every release cycle - i typically skip
> > urgent branch merges after -rc8/-rc9, unless they are very-very-urgent
> > fixes - but they'd still be eligible for -stable.
>
> Ok, that would be good and fine with me.
>
> James, would your script pick this up, or does it need to also pay
> attention to merge commits?

No ... because merge commits should effectively be empty (and when
they're not you can't generate an applyable diff). If I understand the
workflow, the desire is to have the whole branch sent to stable by
tagging the merge commit. That's possible ... it's exactly the same
logic I use in the commit scripts for the SCSI tree, so it should be
possible to extract the logic.

By the looks of the above it's only a few commits, or is it the entire
branch?

> > I've attached the full commit below. The prereq commits are not uptream
> > yet, and they dont carry a -stable backporting tag as the -stable
> > relevance was not anticipated at that point yet. They will all be
> > upstream in the next merge window when Linus merges the relevant tree -
> > and then all these tags become visible to the -stable team's scripts.
> >
> > What do you think about this new -stable tagging variant? To me it looks
> > quite intuitive, less error-prone and it is more informative as well.
> > Furthermore, it gives us some freedom to mark commits as backport
> > candidates later on. I kept them oneliners for the purpose of making
> > them all self-sufficient tags.
>
> I agree.
>
> > ( Sidenote: i wouldnt go as far as to generate null Git commits to mark
> > backports after the fact - this scheme is for a series of commits that
> > get 'completed' - there's usually a final followup commit that can
> > embedd this information. )
>
> That's fine, don't worry about this.

The question is, how important is this?

One of the assumptions behind the current setup is that I assume
backports are independent (so the order of transmission doesn't matter
that much). This isn't always true, but the exceptions tend to get
handled manually. Part of what the above is requesting is an
implementation that starts to care about ordering.

James


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo.DeleteThis@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
James Bottomley

External


Since: Sep 10, 2003
Posts: 82



(Msg. 5) Posted: Tue Nov 10, 2009 9:20 am
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On Tue, 2009-11-10 at 05:20 +0100, Ingo Molnar wrote:
> * Greg KH <gregkh.TakeThisOut@suse.de> wrote:
>
> > > I've attached the full commit below. The prereq commits are not
> > > uptream yet, and they dont carry a -stable backporting tag as the
> > > -stable relevance was not anticipated at that point yet. They will
> > > all be upstream in the next merge window when Linus merges the
> > > relevant tree - and then all these tags become visible to the
> > > -stable team's scripts.
> > >
> > > What do you think about this new -stable tagging variant? To me it
> > > looks quite intuitive, less error-prone and it is more informative
> > > as well. Furthermore, it gives us some freedom to mark commits as
> > > backport candidates later on. I kept them oneliners for the purpose
> > > of making them all self-sufficient tags.
> >
> > I agree.
>
> Ok - thanks for the confirmation - i've pushed out the first such
> commit. (Let me know if there's any problem with it down the line - it
> will be a few weeks, in the next merge window, until it truly
> 'activates' for -stable.)

Can you give me the commit id in your tree so I can run a few tests?

Thanks,

James


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo.TakeThisOut@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
Stefan Richter

External


Since: Jul 26, 2007
Posts: 530



(Msg. 6) Posted: Tue Nov 10, 2009 11:20 am
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

James Bottomley wrote:
[...]
>> On Tue, Nov 10, 2009 at 04:48:31AM +0100, Ingo Molnar wrote:
>> > FYI, today i committed a scheduler performance fix that has a number of
>> > commit prerequisites for -stable integration. Those commits are not
>> > marked -stable.
[...]
>> > we can move this into the Git commit space too, and minimize the
>> > work for the -stable team, via a new -stable tag variant.
[...]
> The question is, how important is this?
>
> One of the assumptions behind the current setup is that I assume
> backports are independent (so the order of transmission doesn't matter
> that much). This isn't always true, but the exceptions tend to get
> handled manually. Part of what the above is requesting is an
> implementation that starts to care about ordering.

More importantly, isn't this against the character of the -stable kernel
branches as _safe and simple_ hotfix branches?

If a fix has a number of prerequisites which ar not -stable fixes
themselves, then it is more than a hint that this fix is not really well
suited for -stable.
--
Stefan Richter
-=====-==--= =-== -=-=-
http://arcgraph.de/sr/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo.TakeThisOut@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
Ingo Molnar

External


Since: Nov 05, 2003
Posts: 2921



(Msg. 7) Posted: Tue Nov 10, 2009 1:20 pm
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

* Linus Torvalds <torvalds DeleteThis @linux-foundation.org> wrote:

>
>
> On Tue, 10 Nov 2009, Ingo Molnar wrote:
> >
> > * James Bottomley <James.Bottomley DeleteThis @HansenPartnership.com> wrote:
> > >
> > > Can you give me the commit id in your tree so I can run a few tests?
> >
> > Let me re-quote what you quoted above:
> >
> > > > > > I've attached the full commit below. [...]
>
> Let me re-quote James:
>
> "Can you give me the commit id in your tree so I can run a few tests?"
>
> He wanted the tree. So that he can run tests on his machinery. His
> machinery that does _not_ take some emailed commit description, but
> works on git trees.
>
> So Ingo, if you want to be snarky about quoting people, at least be
> snarky with a reason.

The commit i quoted had the commit ID on its first line (i did that so
because i anticipated someone wanting to check my tree), but here it is
again plus the (well-known to James) URI of the -tip tree:

> From eae0c9dfb534cb3449888b9601228efa6480fdb5 Mon Sep 17 00:00:00 2001

git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git

Thanks,

Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo DeleteThis @vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
Ingo Molnar

External


Since: Nov 05, 2003
Posts: 2921



(Msg. 8) Posted: Tue Nov 10, 2009 1:20 pm
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

* James Bottomley <James.Bottomley.DeleteThis@HansenPartnership.com> wrote:

> On Tue, 2009-11-10 at 05:20 +0100, Ingo Molnar wrote:
> > * Greg KH <gregkh.DeleteThis@suse.de> wrote:
> >
> > > > I've attached the full commit below. The prereq commits are not
> > > > uptream yet, and they dont carry a -stable backporting tag as the
> > > > -stable relevance was not anticipated at that point yet. They will
> > > > all be upstream in the next merge window when Linus merges the
> > > > relevant tree - and then all these tags become visible to the
> > > > -stable team's scripts.
> > > >
> > > > What do you think about this new -stable tagging variant? To me it
> > > > looks quite intuitive, less error-prone and it is more informative
> > > > as well. Furthermore, it gives us some freedom to mark commits as
> > > > backport candidates later on. I kept them oneliners for the purpose
> > > > of making them all self-sufficient tags.
> > >
> > > I agree.
> >
> > Ok - thanks for the confirmation - i've pushed out the first such
> > commit. (Let me know if there's any problem with it down the line - it
> > will be a few weeks, in the next merge window, until it truly
> > 'activates' for -stable.)
>
> Can you give me the commit id in your tree so I can run a few tests?

Let me re-quote what you quoted above:

> > > > I've attached the full commit below. [...]

Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo.DeleteThis@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
Chris Friesen

External


Since: Nov 10, 2006
Posts: 104



(Msg. 9) Posted: Tue Nov 10, 2009 1:20 pm
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On 11/10/2009 09:52 AM, Stefan Richter wrote:

> More importantly, isn't this against the character of the -stable kernel
> branches as _safe and simple_ hotfix branches?
>
> If a fix has a number of prerequisites which ar not -stable fixes
> themselves, then it is more than a hint that this fix is not really well
> suited for -stable.

Alternately, it's conceivable that the prerequisites were not
in-and-of-themselves candidates for -stable (maybe they didn't do
anything by themselves) but when combined with the final commit the
overall change is suitable for inclusion in -stable.

Chris
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo RemoveThis @vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
Linus Torvalds

External


Since: Jan 22, 2007
Posts: 883



(Msg. 10) Posted: Tue Nov 10, 2009 1:20 pm
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On Tue, 10 Nov 2009, Ingo Molnar wrote:
>
> * James Bottomley <James.Bottomley.RemoveThis@HansenPartnership.com> wrote:
> >
> > Can you give me the commit id in your tree so I can run a few tests?
>
> Let me re-quote what you quoted above:
>
> > > > > I've attached the full commit below. [...]

Let me re-quote James:

"Can you give me the commit id in your tree so I can run a few tests?"

He wanted the tree. So that he can run tests on his machinery. His
machinery that does _not_ take some emailed commit description, but works
on git trees.

So Ingo, if you want to be snarky about quoting people, at least be snarky
with a reason.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo.RemoveThis@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
Greg KH

External


Since: Nov 03, 2006
Posts: 1124



(Msg. 11) Posted: Tue Nov 10, 2009 3:20 pm
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On Tue, Nov 10, 2009 at 09:29:48AM -0500, James Bottomley wrote:
> On Mon, 2009-11-09 at 20:14 -0800, Greg KH wrote:
> > On Tue, Nov 10, 2009 at 04:48:31AM +0100, Ingo Molnar wrote:
> > > FYI, today i committed a scheduler performance fix that has a number of
> > > commit prerequisites for -stable integration. Those commits are not
> > > marked -stable.
> > >
> > > Previously, in similar situations, i solved it by email-forwarding the
> > > prereq commits to stable.RemoveThis@kernel.org. (or by waiting for your 'it does
> > > not apply to -stable' email and figuring out the prereqs then.)
> > >
> > > But we can move this into the Git commit space too, and minimize the
> > > work for the -stable team, via a new -stable tag variant. So with this
> > > new commit i started using a new tagging scheme in the commit itself:
> > >
> > > Cc: <stable.RemoveThis@kernel.org> # .32.x: a1f84a3: sched: Check for an idle shared cache
> > > Cc: <stable.RemoveThis@kernel.org> # .32.x: 1b9508f: sched: Rate-limit newidle
> > > Cc: <stable.RemoveThis@kernel.org> # .32.x: fd21073: sched: Fix affinity logic
> > > Cc: <stable.RemoveThis@kernel.org> # .32.x
> > > LKML-Reference: <1257821402.5648.17.camel.RemoveThis@marge.simson.net>
> > > Signed-off-by: Ingo Molnar <mingo.RemoveThis@elte.hu>
> > >
> > > The tag sequence has the meaning of:
> > >
> > > git cherry-pick a1f84a3
> > > git cherry-pick 1b9508f
> > > git cherry-pick fd21073
> > > git cherry-pick <this commit>
> > >
> > > and i'm wondering whether this tagging scheme is fine with your -stable
> > > scripting, etc.
> >
> > It would work just fine.
> >
> > I only rely on one main script right now, one that runs from James's
> > directory on kernel.org that picks out the "Cc: <stable.RemoveThis@kernel.org>"
> > lines and forwards the full commit message to stable.RemoveThis@kernel.org.
> >
> > Then I have a simple script that I just pass the git commit id and
> > formats it properly for inclusion on the quilt tree for the stable
> > queue. If you list the other git commit ids that are needed as a
> > prerequesite as you did above, that's trivial to also pick out.
> >
> > So I think this is good for me and my workflow.
>
> So I take this to mean that I don't alter my script and you pick out the
> precursors with yours ...

Exactly, it's easier for me that way, as I know the dependancy of what
needs to go before what. And it's just so trivial to feed a git commit
id to my script Smile

> > > A further question is, i can see using this tagging scheme in the future
> > > in merge commits log messages too - will your scripts notice that too?
> >
> > Hm, I don't think we look at merges as there's nothing there to actually
> > commit.
> >
> > > For example if there's a few commits left in tip:*/urgent branches
> > > (tip:sched/urgent, tip:core/urgent, tip:x86/urgent, etc.) by the time
> > > v2.6.32 is released, i will then merge them into tip:sched/core,
> > > tip:core/core, tip:x86/core, etc. - and i could use the merge commit as
> > > a notification area to 'activate' them for -stable backporting via a
> > > merge commit.
> > >
> > > This is how such merge commits would look like:
> > >
> > > Merge branch 'core/urgent' into core/rcu
> > >
> > > Merge reason: Pick up urgent fixes that did not make it into .32.0
> > >
> > > Cc: <stable.RemoveThis@kernel.org> # .32.x: 83f5b01: rcu: Fix long-grace-period race
> > > Signed-off-by: Ingo Molnar <mingo.RemoveThis@elte.hu>
> > >
> > > This is not so rare of a situation as it might seem - for the trees i
> > > maintain it happens in almost every release cycle - i typically skip
> > > urgent branch merges after -rc8/-rc9, unless they are very-very-urgent
> > > fixes - but they'd still be eligible for -stable.
> >
> > Ok, that would be good and fine with me.
> >
> > James, would your script pick this up, or does it need to also pay
> > attention to merge commits?
>
> No ... because merge commits should effectively be empty (and when
> they're not you can't generate an applyable diff). If I understand the
> workflow, the desire is to have the whole branch sent to stable by
> tagging the merge commit. That's possible ... it's exactly the same
> logic I use in the commit scripts for the SCSI tree, so it should be
> possible to extract the logic.
>
> By the looks of the above it's only a few commits, or is it the entire
> branch?

I'm thinking the commit would be the merge, right Ingo? So it would
just be a single commit that has the marker in it.

> > > I've attached the full commit below. The prereq commits are not uptream
> > > yet, and they dont carry a -stable backporting tag as the -stable
> > > relevance was not anticipated at that point yet. They will all be
> > > upstream in the next merge window when Linus merges the relevant tree -
> > > and then all these tags become visible to the -stable team's scripts.
> > >
> > > What do you think about this new -stable tagging variant? To me it looks
> > > quite intuitive, less error-prone and it is more informative as well.
> > > Furthermore, it gives us some freedom to mark commits as backport
> > > candidates later on. I kept them oneliners for the purpose of making
> > > them all self-sufficient tags.
> >
> > I agree.
> >
> > > ( Sidenote: i wouldnt go as far as to generate null Git commits to mark
> > > backports after the fact - this scheme is for a series of commits that
> > > get 'completed' - there's usually a final followup commit that can
> > > embedd this information. )
> >
> > That's fine, don't worry about this.
>
> The question is, how important is this?
>
> One of the assumptions behind the current setup is that I assume
> backports are independent (so the order of transmission doesn't matter
> that much). This isn't always true, but the exceptions tend to get
> handled manually. Part of what the above is requesting is an
> implementation that starts to care about ordering.

No, I'll take care of the ordering myself. Heck, I already have to do
that today with our current setup as we have dependant staging patches
right now. I just want to make sure the merge commits will get picked
up and sent to me so I can then pick the correct git commit ids out of
them.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo.RemoveThis@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
Greg KH

External


Since: Nov 03, 2006
Posts: 1124



(Msg. 12) Posted: Tue Nov 10, 2009 3:20 pm
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On Tue, Nov 10, 2009 at 04:52:22PM +0100, Stefan Richter wrote:
> James Bottomley wrote:
> [...]
> >> On Tue, Nov 10, 2009 at 04:48:31AM +0100, Ingo Molnar wrote:
> >> > FYI, today i committed a scheduler performance fix that has a number of
> >> > commit prerequisites for -stable integration. Those commits are not
> >> > marked -stable.
> [...]
> >> > we can move this into the Git commit space too, and minimize the
> >> > work for the -stable team, via a new -stable tag variant.
> [...]
> > The question is, how important is this?
> >
> > One of the assumptions behind the current setup is that I assume
> > backports are independent (so the order of transmission doesn't matter
> > that much). This isn't always true, but the exceptions tend to get
> > handled manually. Part of what the above is requesting is an
> > implementation that starts to care about ordering.
>
> More importantly, isn't this against the character of the -stable kernel
> branches as _safe and simple_ hotfix branches?
>
> If a fix has a number of prerequisites which ar not -stable fixes
> themselves, then it is more than a hint that this fix is not really well
> suited for -stable.

Not true, we have been doing this kind of thing for quite some time now.
Sometimes it's just a simple "this patch cleans up the code, and the
second one fixes it in an obvious manner" type thing. It is easier for
me and everyone else for us to apply 2 commits to the -stable tree,
instead of rewriting the second patch that actually does the fix and
hope I got it all correct in doing so.

It's also easier to review stuff, which is the most important thing.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo.RemoveThis@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
Ingo Molnar

External


Since: Nov 05, 2003
Posts: 2921



(Msg. 13) Posted: Tue Nov 10, 2009 3:20 pm
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

* Greg KH <gregkh DeleteThis @suse.de> wrote:

> > By the looks of the above it's only a few commits, or is it the
> > entire branch?
>
> I'm thinking the commit would be the merge, right Ingo? So it would
> just be a single commit that has the marker in it.

Correct.

This is really a special case, a small variation of the commit eae0c9d
-stable tagging scheme i outlined in the first mail.

When i merge */urgent branches into the for-linus branch in the merge
window, i cannot change the commits anymore (it would amount to a
rebase), but i have the opportunity to modify the merge commit message
itself. (which is typically a regular merge commit and does not carry
any -stable actionable change itself.)

I already annotate merge commits today - for example:

| commit 43315956509ca6913764861ac7dec128b91eb1ec
| Merge: 9bf4e7f 6beba7a
| Author: Ingo Molnar <mingo DeleteThis @elte.hu>
| Date: Fri Oct 23 08:23:20 2009 +0200
|
| Merge branch 'perf/core' into perf/probes
|
| Conflicts:
| tools/perf/Makefile
|
| Merge reason:
|
| - fix the conflict
| - pick up the pr_*() infrastructure to queue up dependent patch
|
| Signed-off-by: Ingo Molnar <mingo DeleteThis @elte.hu>

Note how i already put a SOB line into the merge commit - i treat every
merge as something that 'had to be done' so they are never arbitrary and
always carry real information.

So my idea was to potentially use the extended -stable notification
scheme in certain merge commits too. Here's a mockup merge commit log:

Merge branch 'sched/urgent' into sched/core

Conflicts:
tools/perf/Makefile

Merge reason:

- resolve the conflict
- queue up urgent fixes for the next merge window

Cc: <stable DeleteThis @kernel.org> # .32.x: a1f84a3: sched: Check for an idle shared cache
Cc: <stable DeleteThis @kernel.org> # .32.x: 1b9508f: sched: Rate-limit newidle
Cc: <stable DeleteThis @kernel.org> # .32.x: fd21073: sched: Fix affinity logic
Signed-off-by: Ingo Molnar <mingo DeleteThis @elte.hu>

Note that the merge commit itself carries no action for -stable: there's
no "Cc: <stable DeleteThis @kernel.org>" line - only 'pointer' lines in the form
of:

Cc: <stable DeleteThis @kernel.org> # .32.x: sha1: title

But ... if you or Linus dislikes this direction of tagging for some
reason i can still do the manual approach as well. It seemed useful to
me though and it would be a natural portion of my workflow.

Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo DeleteThis @vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
Ingo Molnar

External


Since: Nov 05, 2003
Posts: 2921



(Msg. 14) Posted: Tue Nov 10, 2009 3:20 pm
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

* Greg KH <gregkh.DeleteThis@suse.de> wrote:

> > More importantly, isn't this against the character of the -stable
> > kernel branches as _safe and simple_ hotfix branches?
> >
> > If a fix has a number of prerequisites which ar not -stable fixes
> > themselves, then it is more than a hint that this fix is not really
> > well suited for -stable.
>
> Not true, we have been doing this kind of thing for quite some time
> now. Sometimes it's just a simple "this patch cleans up the code, and
> the second one fixes it in an obvious manner" type thing. It is
> easier for me and everyone else for us to apply 2 commits to the
> -stable tree, instead of rewriting the second patch that actually does
> the fix and hope I got it all correct in doing so.
>
> It's also easier to review stuff, which is the most important thing.

Yeah. This new tagging scheme doesnt really allow anything 'new' per se
- it just helps the existing practice some more. All these commits were
-stable candidates anyway, in exactly the same order - the only
difference the new tagging scheme adds here is a more organized,
in-upsream-Git way of communicating it to you.

This is also easier and less error prone for me than using email: i can
do all the -stable tagging when i create a commit - or if i see that a
commit has prereqs and those should be in -stable too. In those
situations i check out the last stable kernel version, and cherry-pick
the prereqs and the target commit, to see that it cherry-picks without
conflicts.

But i cannot send you an email to stable.DeleteThis@kernel.org just yet: as i
havent fully tested the last commit yet, and have not pushed it out yet.
The commit ID is not stable yet.

So without the in-commit tagging, i'd have to remember to send you an
email in an hour (or in a day - whenever testing is done) - and that is
error prone and easy to forget. The prereqs might be lost, etc. It's
better to do this all in one well-focused moment of time, gather the
information and mention it in the changelog.

Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo.DeleteThis@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
Junio C Hamano

External


Since: Jun 17, 2007
Posts: 19



(Msg. 15) Posted: Tue Nov 10, 2009 3:20 pm
Post subject: Re: [RFC] new -stable tag variant, Git workflow question [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Ingo Molnar <mingo.DeleteThis@elte.hu> writes:

> Yeah. This new tagging scheme doesnt really allow anything 'new' per se
> - it just helps the existing practice some more. All these commits were
> -stable candidates anyway, in exactly the same order - the only
> difference the new tagging scheme adds here is a more organized,
> in-upsream-Git way of communicating it to you.

I am just a bystander, but if it were truly in-upstream-git way, wouldn't
you be forking a branch from the tagged target release (the latest of
2.6.32.X), and queuing only the changes meant for -stable to it, and
giving the name of the branch to git people and sending out patches from
that branch for e-mailed review and application?

There won't be any special tagging required, only a dedicated branch.

Or am I missing something?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo.DeleteThis@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to top
Login to vote
Display posts from previous:   
Related Topics:
A simpler variant on sys_indirect? - I was just thinking, while sys_indirect is an interesting way to add features to a system call, the argument marshallin...

[stable] Wanted: Allow adding new device IDs during the -s.. - I'd like to propose we allow adding new device IDs as part of the -stable process, but only under certain conditions: ....

[stable] libata spindown patches for 2.6.21-stable - Should we put these patches in 2.6.21-stable? Gentoo developers did a full backport: ..

[patch-stable 0/3] Futex fixes for stable - Hi folks, the following patch series is a backport of the futex fixups for stable. Patch 1 and 2 are identical to the...

ReiserFS corruption with 2.6.19 (Was 2.6.19 is not stable .. - Okay, I'm having a really bad day (or week) and I want to apologize for the rantish email. I just went back into the..

GIT and the current -stable - Good day. Stumbling around with git here. I'd like to use git to efficiently track the current -stable as well as..
       Soft32 Home -> Linux -> Kernel All times are: Pacific Time (US & Canada) (change)
Goto page 1, 2, 3
Page 1 of 3

 
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Categories:
 Windows
  Linux
 Mac
 PDA


[ Contact us | Terms of Service/Privacy Policy ]