19:02:05 <wumpus> #startmeeting 
19:02:05 <core-meetingbot> Meeting started Thu May 20 19:02:05 2021 UTC.  The chair is wumpus. Information about MeetBot at https://bitcoin.jonasschnelli.ch/ircmeetings.
19:02:05 <core-meetingbot> Available commands: action commands idea info link nick
19:02:13 <wumpus> meshcollider: you're right
19:02:16 <jnewbery> hi
19:02:19 <hebasto> hi
19:02:21 <sipsorcery> hi
19:02:22 <achow101> hi
19:02:26 <meshcollider> hi
19:02:27 <wumpus> #bitcoin -core-dev Meeting: achow101 aj amiti ariard bluematt cfields Chris_Stewart_5 digi_james dongcarl elichai2 emilengler fanquake fjahr gleb glozow gmaxwell gwillen hebasto instagibbs jamesob jb55 jeremyrubin jl2012 jnewbery jonasschnelli jonatack jtimon kallewoof kanzure kvaciral lightlike luke-jr maaku marcofalke meshcollider michagogo moneyball morcos nehan NicolasDorier paveljanik
19:02:29 <wumpus> petertodd phantomcircuit promag provoostenator ryanofsky sdaftuar sipa vasild wumpus
19:02:39 <jonatack> hi
19:02:39 <michaelfolkson> hi
19:02:51 <fjahr> hi
19:03:34 <wumpus> three proposed meetings for today: moving to oftc or libera.chat (aj), windows code signing certificate update (achow101), remove fuzzer from CI jobs (jnewbery)
19:03:34 <sipa> bhi
19:03:44 <wumpus> any last minute topics?
19:04:33 <wumpus> #topic High priority for review
19:04:33 <core-meetingbot> topic: High priority for review
19:04:55 <wumpus> https://github.com/bitcoin/bitcoin/projects/8 : currently 9 blockers, no bugfixes, no chasing concept ACK
19:04:59 <provoostenator> https://github.com/bitcoin-core/gui/pull/4 HWW GUI support would be cool, though technically doesn't block anything
19:05:05 <wumpus> anything to add/remove or that is (almost) ready for merge
19:05:33 <wumpus> is it possible to add a GUI PR there?
19:05:59 <wumpus> unfortunately i don't think so
19:06:36 <hebasto> it should work to add with full link
19:06:42 <wumpus> let me try
19:08:21 <wumpus> right i could add it through a 'note'
19:08:51 <hebasto> wumpus: provoostenator: thanks!
19:09:35 <wumpus> amything else for high prio?
19:10:31 <wumpus> #topic Moving to oftc or libera.chat (aj)
19:10:32 <core-meetingbot> topic: Moving to oftc or libera.chat (aj)
19:11:21 <wumpus> i'm not sure how much of a discussion this is anymore, fwiw: we've already reserved the namespace and channels in libera.chat
19:11:49 <wumpus> it makes sense to register your nickname there if you haven't done so yet (works the same as here, with nickserv)
19:12:21 <hebasto> how long it would take to switch from freenode?
19:12:54 <wumpus> i don't think there's any work left to be done?
19:13:17 <michaelfolkson> This is still up in the air right? We're going to wait until dust has settled to decide rather than move in say the next week or two?
19:13:20 <wumpus> okay, the merges bot isn't there yet, and we need to update the bitcoincore.org website
19:13:35 <BlueMatt> I dont think it should be a question on if things should move, there's not really any doubt that it should have moved already, only question is libera, oftc, or something else
19:14:45 <michaelfolkson> BlueMatt: Because new owner is "malicious"? I haven't been following it that closely. Are there any devs still left on the Freenode side?
19:14:52 <wumpus> from what i've seen there is pretty broad agreement to move
19:15:25 <hebasto> is current lack of tor support on libera a blocker?
19:15:29 <wumpus> michaelfolkson: it's really fishy what happened, legal threats against admins etc
19:16:13 <wumpus> https://gist.github.com/joepie91/df80d8d36cd9d1bde46ba018af497409
19:16:14 <achow101> oftc seems to be a bit harder to register with since they don't support sasl
19:17:04 <hebasto> ^ without public plans to add such support?
19:17:05 <BlueMatt> michaelfolkson: the people behind the freenode acquisition have been known hostile to bitcoin for years, I kinda suggested moving, but wasnt worth the effort, this seems like a good excuse more than anything.
19:17:25 <b10c_> hi
19:17:31 <wumpus> hebasto: libera is planning to add tor support at least they mention so on their page
19:17:34 <michaelfolkson> BlueMatt: Ok thanks
19:17:48 <sipa> today i was briefly able to connect to libera over tor
19:17:59 <hebasto> nice
19:18:17 <BlueMatt> (the actual freenode admins have been fairly friendly to bitcoin stuff, at least after a *ton* of effort on the part of some of the #bitcoin mods, most of those admins have moved to libera now)
19:19:22 <provoostenator> I registered provoostenator on the other side :-)
19:20:12 <murch> provoostenator: Well, that settles it then. ;)
19:20:43 <wumpus> it's wise to register your name there, to prevent it from being grabbed by someone else
19:20:57 <provoostenator> And then confirm it here.
19:21:07 <wumpus> yes
19:21:07 <michaelfolkson> I suppose my second and final question would be there isn't the possibility of a reversal and everyone makes up and goes back to Freenode right? This often seems to happen in scenarios like this. A protest which results in a new agreement
19:21:19 <hebasto> am I understand correctly hat there is a consensus to move from freenode?
19:21:22 <provoostenator> (depending on *how* hostile the IRC overloards are here of course)
19:21:48 <hebasto> * that
19:22:39 <achow101> it's prudent to maintain accounts on multiple networks so that switching is merely changing a url
19:23:04 <wumpus> michaelfolkson: i sincerely doubt it (but cannot really go into details)
19:23:31 <wumpus> read the gist i posted for some information
19:23:45 <BlueMatt> michaelfolkson: I think if we find much wrong with libera, we will move to oftc or elsewhere, there's about zero chance we end up back on freenode
19:24:13 <michaelfolkson> wumpus BlueMatt: Ok, sounds like it is just a question of timing then
19:25:03 <jonatack> https://libera.chat/guides/registration
19:25:37 <hebasto> my vote is for libera
19:25:59 <achow101> libera is fine with me
19:26:04 <wumpus> same
19:26:18 <michaelfolkson> Yeah libera seems obvious choice
19:26:18 <jonatack> yup
19:27:21 <provoostenator> Who wants to register satoshi to keep out the name squatters,  and receive lots of legal threats? :-)
19:27:59 <wumpus> so the remaining question is timeframe
19:28:19 <wumpus> where is next week's IRC meeting?
19:28:52 <provoostenator> It could make sense to do that on libera, but maybe ask right before the meeting if anyone objects.
19:28:55 <sipa> maybe let's plan to have next week's meeting be the last one here?
19:29:02 <sipa> unless there are exigent circumstances
19:29:15 <achow101> There are also all of the side meetings (e.g. wallet, p2p)
19:29:39 <jnewbery> those should follow the main meeting
19:29:42 <ariard> hi
19:29:46 <michaelfolkson> Yeah it is good to keep people informed and not move quickly unless forced to (I think)
19:30:06 <jonatack> next wallet meeting is tomorrow (here, presumably)
19:30:08 <michaelfolkson> Last Core dev meeting here next week sounds good to me (and gives time for people to get set up)
19:30:20 <wumpus> sgtm
19:30:24 <hebasto> agree
19:30:27 <achow101> ack
19:30:40 <jonatack> i'm fine to switch as soon as people want to
19:31:22 <wumpus> #topic Windows code signing certificate update (achow101)
19:31:22 <core-meetingbot> topic: Windows code signing certificate update (achow101)
19:31:22 <hebasto> how people will be informed about moving?
19:31:40 <wumpus> hebasto: we can set the topic here i guess
19:31:50 <hebasto> wumpus: ok
19:31:50 <achow101> It looks like we will be getting the windows code signing certificate shortly, hopefully within in the next week
19:31:57 <wumpus> achow101: great news!!!
19:32:11 <sipa> awesöme
19:32:18 <wumpus> hebasto: and we'll need to badger people talking here for a while to move maybe :)
19:32:21 <achow101> We have created Bitcoin Core Code Signing LLC registerd in Delaware and the cert will be issued to this new LLC
19:32:46 <wumpus> hebasto: then at some point maybe set it to +m
19:32:59 <achow101> it's been validated already, so all that's left is waiting for Digicert to issue the cert itself
19:33:06 <wumpus> achow101: glad to hear that
19:33:37 <achow101> that's all
19:33:51 <provoostenator> Nice!
19:34:21 <wumpus> #topic Remove fuzzer from CI jobs (jnewbery)
19:34:21 <core-meetingbot> topic: Remove fuzzer from CI jobs (jnewbery)
19:35:33 <jnewbery> hi!
19:36:07 <jnewbery> It seems that cirrus CI very frequently times out after two hours on the job that runs the fuzz corpus
19:36:35 <jnewbery> and that causes the CI to show as failed for PRs that don't actually have any problems
19:36:56 <wumpus> yes, it happens intermittently but quite often
19:37:13 <jnewbery> There are two problems here: 1. a run time of two hours is a really slow feedback loop, which is terrible for productivity
19:37:24 <wumpus> i wonder if there are any specific cases that are so slow or it's necessarily like this
19:37:40 <sipa> i think the fuzz test sets are also just too big
19:37:41 <provoostenator> Is it possible to a much more limited fuzz? Just to catch really obvious mistakes?
19:37:47 <wumpus> two hour is long yes
19:37:49 <jnewbery> 2. a CI that has false failures reduces people's trust in the system and hides actual failures
19:37:57 <jonatack> protip, push on sunday
19:38:03 <jnewbery> I personally don't think we should be running the fuzz corpus on PRs
19:38:16 <sipa> jnewbery: you mean just run it on master?
19:38:21 <sipa> that'd be fine by me
19:38:40 <ajonas> so oss-fuzz does have a ci intergration
19:38:50 <jnewbery> if the code that the corpus is testing hasn't changed, then you're just running the same code paths as every other path. If it has changed, then the corpus isn't going to get good coverage (since it's optimized for the old code)
19:38:51 <ajonas> no idea what the run time would be but it's something to maybe consider
19:38:55 <jnewbery> yes, just run on master
19:39:17 <jnewbery> I feel like a CI is not the right place for fuzzing
19:39:26 <jonatack> at the same time, the fuzz CI helpfully catches when you forget to update the fuzzers for your changes
19:39:49 <sipa> jonatack: we can build the fuzz code without running it
19:39:50 <jnewbery> jonatack: we should still build the fuzz binaries
19:39:53 <hebasto> ^ we have fuzz binary by default
19:40:00 <michaelfolkson> provoostenator: I don't think we're at a point where we can draw a clear divide between obvious fuzzing and non obvious fuzzing (but someone can correct me if wrong)
19:40:31 <sipa> another possibility is just running the a small random subset of fuzz inputs
19:40:58 <sipa> but i think i agree with jnewbery that in general, (PR) CI is not the right place for fuzzing
19:41:14 <jnewbery> sorry, typo in my earlier statement: s/the same code path as every other path/the same code path as every other branch/
19:41:14 <ajonas> The fuzzer is not the only thing that's slow
19:41:27 <sipa> well, what we're doing isn't even actual fuzzing, it's running unit tests that are automatically derived from fuzzing corpus :)
19:41:36 <ajonas> it will knock the feedback loop down some but it's not the only issue if that's what the goal is
19:42:05 <sipa> ajonas: what else is?
19:42:10 <jnewbery> sipa: not really unit tests, since they don't necessarily assert that the behaviour is correct.
19:42:23 <jnewbery> they just try to hit path coverage
19:42:23 <sipa> jnewbery: true
19:42:24 <glozow> the fuzzer is the one that’s timing out a lot though right?
19:43:07 <ajonas> the macbuild actually takes longer on average
19:43:18 <jonatack> and there is still a fuzzer run on bitcoinbuilds atm
19:43:18 <ajonas> same with tsan
19:43:23 <wumpus> glozow: yes it's why your testmempoolaccept PR is failing the CI
19:43:30 <michaelfolkson> glozow: Nothing else takes two hours as far as I know :)
19:43:31 <glozow> yes :’(
19:43:53 <jonatack> michaelfolkson: i have the impression that it can vary quite a bit
19:44:00 <jnewbery> my proposal would be to remove the fuzz corpus testing from our CI. I also think separately we should all aim to drive down the CI time - fast feedback loops are really important
19:45:00 <ajonas> sorry that's wrong https://usercontent.irccloud-cdn.com/file/9vK4c0d1/Screen%20Shot%202021-05-20%20at%203.44.36%20PM.png
19:45:07 <ajonas> this is the last 7 days
19:45:08 <wumpus> left by itself the time taken by test frameworks seems to be forever increasing :)
19:45:16 <jnewbery> wumpus: yes :(
19:45:42 <jonatack> ajonas: thanks!
19:45:59 <ajonas> that's just the queue
19:46:11 <ajonas> https://usercontent.irccloud-cdn.com/file/8Cv7Yv85/Screen%20Shot%202021-05-20%20at%203.45.46%20PM.png
19:46:20 <ajonas> that's the build duration
19:46:36 <jnewbery> ajonas: is that data pulled from cirrus?
19:46:44 <ajonas> yes
19:47:18 <michaelfolkson> So ACK on removing fuzzer from CI. Does there need to be a separate conversation on the increasing time taken by test frameworks? Or is it inevitable that slowly goes up over time?
19:47:40 <jnewbery> it's good to have that data
19:47:53 <sipa> there is always the possibility of splitting up a very slow CI target in two
19:47:54 <lightlike> has there ever been an instance where the fuzzer corpus run found a bug on a PR?
19:48:14 <jnewbery> It doesn't match what I'm seeing on some PRs. For example 20833 has failed many times in a row because the fuzzer times out after two hours
19:48:23 <jnewbery> that seems unlikely if the mean really is 40 minutes
19:48:32 <wumpus> yes, i'm also fine with removing the fuzzer from the CI, i think it was mostly useful to that the fuzz tests got compiled, but that's the default now
19:48:41 <jnewbery> unless there's something in that PR that causes the fuzzer job time to triple
19:50:01 <sipa> jnewbery: maybe it's only the mean of successful jobs?
19:50:24 <ajonas> definitely possible
19:50:27 <ajonas> can check on that
19:50:39 <michaelfolkson> lightlike: I'm not sure from the CI on a PR. But many bugs found by fuzzing generally https://github.com/bitcoin-core/bitcoin-devwiki/wiki/Fuzz-Trophies
19:50:58 <glozow> this might be a dumb question but can we just run fuzz tests that are changed in the pr?
19:51:08 <sipa> glozow: not sure how we'd do that
19:51:32 <glozow> me neither ðŸ˜Â
19:51:49 <sipa> michaelfolkson: those are all due to actual fuzzing
19:52:05 <michaelfolkson> sipa: Ok
19:52:06 <jnewbery> I'm also not sure of the value of running a fuzz corpus derived from an old branch on the new branch
19:52:20 <sipa> jnewbery: it could detect certain regressions
19:52:53 <jnewbery> either the code is the same and you're running the same executions, or the code is different and the fuzz corpus won't penetrate very deeply into the new code paths
19:53:10 <sipa> jnewbery: i think that's a bit unnuanced
19:53:10 <jnewbery> maybe I just don't understand the idea of corpuses
19:53:33 <sipa> you're certainly right for certain types of changes, but not all
19:53:33 <jnewbery> sipa: probably :)
19:54:12 <ajonas> I don’t know think a slow and fast makes sense.  Fanquake also brought up that much of the Mac build is brew installing stuff. We can do better.
19:54:39 <sipa> in any case, i agree that there is little value in running these tests in CI on every PR
19:54:39 <ajonas> That was garbled. A slow and fast test separation makes sense.
19:54:44 <jnewbery> I think there's probably more analysis to be done. Knowing how many regressions were caught by running the fuzz corpus would be really interesting
19:55:18 <sipa> jnewbery: a useful test could be trying to reintroduce an old bug that was found by fuzzing
19:55:19 <jnewbery> do we have a CI job on the master branch after every merge? Would that be a better place for this test?
19:55:26 <sipa> (but without straight up reverting the code)
19:55:45 <sipa> jnewbery: i think so
19:55:50 <jnewbery> I know we used to with Travis. I'm not very familiar with the cirrus config
19:55:59 <sipa> hmm, not sure either
19:56:02 <sipa> MarcoFalke: ^
19:56:15 <hebasto> yes we have
19:56:46 <sipa> https://cirrus-ci.com/build/5945300410957824 e.g.
19:57:09 <sipa> so it seems we indeed do
19:57:12 <jnewbery> https://cirrus-ci.com/github/bitcoin/bitcoin/master
19:57:51 <jnewbery> it looks like many of those jobs have timed out after 2 hours as well :(
19:58:13 <sipa> well ne reason why those would be faster than the CI ones
19:58:15 <sipa> *no
19:58:26 <jnewbery> I guess there must be huge variance in the fuzz job time
19:59:44 <wumpus> meeting ending in a bit
19:59:54 <ajonas> Chaincode has a dashboard that we can clean up and make available.
20:00:04 <ajonas> Happy to do that if there is interest.
20:00:23 <jnewbery> ajonas: thanks for all your work looking into this. I think it'll be really useful once we have reliable, fast CI
20:00:27 <jnewbery> makes a huge difference
20:00:57 <ajonas> Marco has done great work getting the functional test suite in shape.
20:01:06 <jnewbery> thanks MarcoFalke!
20:01:10 <ajonas> Lots still to do but help is appreciated.
20:01:21 <jnewbery> I don't have anything else to add
20:01:38 <jnewbery> (in this meeting)
20:01:40 <wumpus> #endmeeting