This will be pretty easy, no?

The title speaks of failures (so I understand it as automated tests results), but the description of comments (what we used in Bodhi1 and want to get rid of in Bodhi2). So I'm not totally sure what you mean here, but I assume you mean you want to send an email for failures in automated tests results.

This would be great to have in Bodhi. Until this is implemented, Taskotron will probably continue posting comments into Bodhi (once I fix it, should be soon) to make sure maintainers are informed.

The implementation might be a bit more complex, if you try to avoid spamming maintainers too much. We have some logic in Taskotron, but it's an ugly hack. Still, some good ideas might be:

• inform only about failures, not about other states
• only inform about failures if the state changed (pass->fail), not when it stayed the same (fail->fail)
• you might decide to inform about fail->fail once in X days, to remind maintainers of the problem (we used X=3 in Taskotron). I'm not sure whether this is a good idea or not, though.
• inform about failures just in the "critical" tests (depcheck, upgradepath), not in the rest (rpmlint, more to come). If we informed about failures in rpmlint, people would tear us into pieces (most packages fail that) and we currently have no option to whitelist selected rpmlint errors (many of them are really nitpicks and often wrong, like supposed spelling errors).
• you can try to wait and group more results into one email (e.g. not inform separately about depcheck i386 and depcheck x86_64, but wait for both to complete and inform then). However, currently it's not really possible to wait for all of depcheck32/64+upgradepath, because upgradepath currently runs only for packages in updates-pending, nothing else. With Bodhi2, we might be finally able to improve on that, but that's the current state.

What about relying 100% on fedmsg for this?

• Taskotron send messages for test results
• Bodhi queries datagrepper for test results for an update and integrates it in the UI
• FMN has rules added to notification maintainers about test results (by default only failure) from taskotron

If taskotron provides enough information about the results, FMN can filter by "critical tests", do the grouping. Reading the list, the only thing I can think of that might be problematic for FMN is the 'do not inform upon fail->fail' unless taskotron knows this information and thus can include it in the message.

Closing this issue, since Taskotron comments trigger email notifications. It does not trigger fedmsgs right now, because I believe QA folks want taskotron to do this itself.

In the future we could look into hooking up various types of test-specific FMN rule triggers, or something of the sort.

No no no no no. We send Taskotron comments into Bodhi because this is not implemented, not the other way round :-)

We would love to get rid of posting Bodhi comments and have native notifications through Bodhi. Can we reopen this please?

Taskotron now emits fedmsgs for results. AFAIK, we try to avoid emitting duplicated results (the same build/update with the same test outcome). Please reopen this ticket, thanks.

Re-opening by request from @kparal in #fedora-admin.

If we disable sending comments from Taskotron (which we really want to do), does Bodhi have enough information to send e-mail notifications about failed checks to maintainers? I know that Bodhi checks resultsdb for check results but I am not sure if the results are stored persistently in Bodhi. If not, Taskotron sends fedmsgs which can be used for that.

What are the options?

Thanks!

I know that Bodhi checks resultsdb for check results but I am not sure if the results are stored persistently in Bodhi.

They are not stored. :)

Is there any reason to have Bodhi involved in the emails at all?

Can we just have taskotron -> fedmsg -> FMN -> maintainers?

I have described my concerns in https://phab.qadevel.cloud.fedoraproject.org/D691#13273 . I'd love to learn that I'm completely wrong and the sun is shining over the rainbows :) Or maybe that I care too much about spamming the maintainers and it's not such a big deal.

OK, there are lots of things still to be done here. It will be helpful to enumerate them. For posterity, I just submitted fedora-infra/fedmsg_meta_fedora_infrastructure#349 which is relevant here.

FYI, that fedmsg.meta update is going out now. it should be in staging this afternoon.

Also, @kparal enumerated the rest of the things that need to be done as blockers on this epic ticket -> https://phab.qadevel.cloud.fedoraproject.org/T627

OK, we should be good to go here now. Please re-open if that is not the case.