Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Obsolete/eject older testing updates before mashing. #94

Open
lmacken opened this issue Sep 24, 2014 · 6 comments
Open

Obsolete/eject older testing updates before mashing. #94

lmacken opened this issue Sep 24, 2014 · 6 comments
Labels
Composer Issues related to the composer High priority These issues are higher priority than normal

Comments

@lmacken
Copy link
Contributor

lmacken commented Sep 24, 2014

We currently hit race conditions when an update is on it's way to testing, and the maintainer submits a newer version during that time. This can lead to the older update getting ignored after it is pushed, and when the newer one goes to stable, the older one is left in testing with garbage-collected signatures.

So, post-tagging pre-mash we should add some additional logic to obsolete the stale testing updates.
@lmacken lmacken added the Critical We can't go on living in this sqalor, drop everything and fix it! label Sep 24, 2014
@lmacken
Copy link
Contributor Author

lmacken commented Sep 24, 2014

Another potential solution could be to only perform the obsoletion during a push, along with everything else. This way, A would go to testing, B would get submitted as well during the process. Then at the next push A would get obsoleted, and B would go to testing.

@ralphbean ralphbean added the Composer Issues related to the composer label Aug 21, 2015
@lmacken lmacken changed the title Obsolete older testing updates before mashing. Obsolete/eject older testing updates before mashing. Sep 2, 2015
@lmacken lmacken mentioned this issue Sep 2, 2015
Closed
@lmacken
Copy link
Contributor Author

lmacken commented Sep 2, 2015

I like @ralphbean's idea to eject the older updates from the mash, however
I don't think we want to potentially eject a giant multi-build update in this
case. This happened recently with a big boost update that had an older build than a single boost update.

@lmacken
Copy link
Contributor Author

lmacken commented Sep 2, 2015

(from issue #115): We need to ensure that bodhi does not let koji tag multiple builds for the same package out of order during a push.

RIght now if a push contains 2 different builds for the same package, bodhi will order them and call the koji.addTag in the correct order, however the multicall does not have any guarantee in terms of execution ordering. So, we'll need to do multiple passes, tagging the old ones first, waiting for completion, then moving on.

@lmacken lmacken mentioned this issue Sep 2, 2015
Closed
@ralphbean
Copy link
Contributor

We merged #420. Does it not cover this?

@lmacken
Copy link
Contributor Author

lmacken commented Sep 3, 2015

#420 covers when there are tag problems, where as with this scenario everything gets tagged properly, but not necessarily in the correct order.

@lmacken lmacken added this to the 2.1.0 milestone Jul 18, 2016
@bowlofeggs bowlofeggs self-assigned this Aug 29, 2016
@bowlofeggs
Copy link
Contributor

It sounds like releng hit this problem again today:

[11:34] <sgallagh> adamw: How did kernel-4.8.0-0.rc2.git3.1.fc25 get into Alpha without getting pushed stable? Beyond that, how did it get into Alpha *or* updates-testing without being in a Bodhi update?
[11:34] <adamw> sgallagh: https://bodhi.fedoraproject.org/updates/FEDORA-2016-0dd1a509c8
[11:36] <sgallagh> adamw: Weird, it doesn't show up on https://bodhi.fedoraproject.org/updates/?like=kernel
[11:36] <adamw> huh
[11:37] <sgallagh> adamw: I'm also not sure why it shows up in Koji tagged with both F25 and F25-updates-testing 
[11:38] <sgallagh> (Mostly I discovered this because I updated to F25 Alpha today on my primary machine and the kernel didn't boot until I pulled that one from u-t...)
[11:39] <kalev> that's normal for Branched, bodhi never untags builds from updates-testing when they move to stable
[11:39] <pbrobinson> sgallagh: I thought it had been pushed stable
[11:39] <kalev> Branched bodhi has a special config so that it keeps builds tagged as both
[11:40] <adamw> https://dl.fedoraproject.org/pub/fedora/linux/development/25/Everything/x86_64/os/Packages/k/ does show it...
[11:40] <sgallagh> OK, that seems odd, but whatever
[11:40] <adamw> oh hmm, no it doesn't
[11:40] <pbrobinson> sgallagh: bodhi doesn't untag from updates-testing when in branched (ie u-t -> f25) it's some weird arse ism that we clean up manually
[11:40] <adamw> that has rc1.git3.1
[11:40] <adamw> nirik: dgilmore: mboddu: ahoy
[11:40] <sgallagh> adamw: Right, and that one failed to boot.
[11:40] <pbrobinson> kalev: it's not a special config, it's a bug I believe :)
[11:41] <sgallagh> I sense problems
[11:42] <dgilmore> adamw: yes sir
[11:42] <sgallagh> adamw: Is that different from what's on the actual installed RCs?
[11:42] <adamw> dgilmore: looks like https://bodhi.fedoraproject.org/updates/FEDORA-2016-0dd1a509c8 didn't push properly
[11:42] <adamw> dgilmore: as in, it's not in https://dl.fedoraproject.org/pub/fedora/linux/development/25/Everything/x86_64/os/Packages/k/
[11:42] <dgilmore> sgallagh: bodhi tags builds into both F25 and F25-updates-testing
[11:43] <adamw> or maybe it got gazumped? does that still happen?
[11:43] <sgallagh> adamw: gazumped?
[11:43] <mboddu> adamw: hello
[11:43] <adamw> sigh, yes, that's what happened.
[11:43] <adamw> https://bodhi.fedoraproject.org/updates/FEDORA-2016-1784e6a050 got pushed over the top of it
[11:43] <adamw> and apparently we *still* don't protect against that
[11:44] <adamw> dgilmore: can you fix it?
[11:44] <dgilmore> adamw: Mon Aug 22 17:07:35 2016 kernel-4.8.0-0.rc2.git3.1.fc25 tagged into f25 by bodhi [still active]
[11:44] <dgilmore> Sat Aug 27 05:10:30 2016 kernel-4.8.0-0.rc1.git3.1.fc25 tagged into f25 by bodhi [still active]
[11:44] <sgallagh> adamw: But the install media has the right version?
[11:44] <dgilmore> it pushed the older build after
[11:44] <dgilmore> sgallagh: it does 
[11:44] <sgallagh> Because https://dl.fedoraproject.org/pub/fedora/linux/development/25/Server/x86_64/os/Packages/k/ has me nervous
[11:44] <dgilmore> different processes and measures 
[11:44] <sgallagh> Which means the netinstall probably *doesn't*
[11:44] <dgilmore> sgallagh: ?
[11:45] <adamw> sgallagh: the actual alpha has the right one.
[11:45] <adamw> netinstalls will be getting the wrong one currently, yes.
[11:45] <sgallagh> adamw: Right, but anyone installing from netinstall media right now is going to end up with the older one, isn't it?
[11:45] <adamw> sure.
[11:46] <adamw> that's why i asked dgilmore if he can fix it.
[11:46] <sgallagh> Which is probably the lion's share of Server installs. Whee.
[11:46] <dgilmore> adamw: yes its fixable
[11:47] <dgilmore> adamw: koji latest-build f25 kernel
[11:47] <dgilmore> Build                                     Tag                   Built by
[11:47] <dgilmore> ----------------------------------------  --------------------  ---------------- 
[11:47] <dgilmore> kernel-4.8.0-0.rc2.git3.1.fc25            f25                   jforbes
[11:47] <dgilmore> tomorrows branched will be fixed 
[11:48] <sgallagh> /me is glad he upgraded today.
[11:49] <adamw> i thought we had a ticket for it somewhere.
[11:49] <dgilmore> adamw: indeed
[12:26] <bowlofeggs> adamw, dgilmore: if you can find a ticket filed about this kernel thing, can you send it my way? otherwise, feel free to file a new one, or feel free for me to file one for you ☺
[12:26] <bowlofeggs> just let me know
[12:26] <bowlofeggs> i'll make sure it's high priority
[12:28] <nirik> https://github.com/fedora-infra/bodhi/issues/94 perhaps

My summary interpretation of the problem is that updates don't get obsoleted when newer updates are being pushed to stable, which can cause the older package to remain open as an update.

@bowlofeggs bowlofeggs removed this from the 2.3 milestone Oct 28, 2016
@bowlofeggs bowlofeggs added High priority These issues are higher priority than normal and removed Critical We can't go on living in this sqalor, drop everything and fix it! labels Feb 27, 2017
@bowlofeggs bowlofeggs removed their assignment Jun 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Composer Issues related to the composer High priority These issues are higher priority than normal
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lmacken @ralphbean @bowlofeggs