19:00:21 <wumpus> #startmeeting 19:00:22 <core-meetingbot> Meeting started Thu Dec 17 19:00:21 2020 UTC. The chair is wumpus. Information about MeetBot at https://bitcoin.jonasschnelli.ch/ircmeetings. 19:00:22 <core-meetingbot> Available commands: action commands idea info link nick 19:00:26 <jonasschnelli> hi 19:00:29 <hebasto> hi 19:00:31 <bitcoin-git> [bitcoin] theStack opened pull request #20689: contrib: replace binary verification script verify.sh with python rewrite (master...202012-contrib-replace-verify-binaries-script-with-python) https://github.com/bitcoin/bitcoin/pull/20689 19:00:47 <wumpus> #bitcoin -core-dev Meeting: achow101 aj amiti ariard bluematt cfields Chris_Stewart_5 digi_james dongcarl elichai2 emilengler fanquake fjahr gleb gmaxwell gwillen hebasto instagibbs jamesob jb55 jeremyrubin jl2012 jnewbery jonasschnelli jonatack jtimon kallewoof kanzure kvaciral lightlike luke-jr maaku marcofalke meshcollider michagogo moneyball morcos nehan NicolasDorier paveljanik 19:00:49 <wumpus> petertodd phantomcircuit promag provoostenator ryanofsky sdaftuar sipa vasild wumpus 19:00:54 <jonatack> hi 19:00:55 <sdaftuar> hi 19:00:56 <achow101> hi 19:01:02 <provoostenator> hi 19:01:05 <fjahr> hi 19:01:21 <jamesob> hi 19:01:29 <wumpus> one proposed meeting for this week: rc4 status, macos codesigning fix (wumpus) 19:02:22 <wumpus> the other is more a PSA: i don't think it makes sense to do meetings 2020-12-24 & 2020-12-31, so the next one will be in 2021! 19:02:48 <jonasschnelli> agree 19:02:53 <meshcollider> hi 19:02:53 <achow101> ack 19:03:06 <wumpus> #topic High priority for review 19:03:06 <core-meetingbot> topic: High priority for review 19:03:13 <sipa> hi 19:03:14 <jonasschnelli> I have added #15946 19:03:17 <gribble> https://github.com/bitcoin/bitcoin/issues/15946 | Allow maintaining the blockfilterindex when using prune by jonasschnelli ÷ Pull Request #15946 ÷ bitcoin/bitcoin ÷ GitHub 19:03:20 <jonasschnelli> (to hiprio) 19:03:26 <wumpus> https://github.com/bitcoin/bitcoin/projects/8 10 blockers, 2 chasing concept ACK 19:03:39 <wumpus> jonasschnelli: ok 19:04:23 <wumpus> otherwise, anything to add/remove or that is ready for merge? 19:04:28 <jonatack> can drop #20546, i'll likely break it down into smaller ones 19:04:31 <gribble> https://github.com/bitcoin/bitcoin/issues/20546 | policy, wallet, refactor: check for non-representable CFeeRates by jonatack ÷ Pull Request #20546 ÷ bitcoin/bitcoin ÷ GitHub 19:04:36 <jonasschnelli> #15946 is basically an lnd stopper for pruned peers 19:04:38 <gribble> https://github.com/bitcoin/bitcoin/issues/15946 | Allow maintaining the blockfilterindex when using prune by jonasschnelli ÷ Pull Request #15946 ÷ bitcoin/bitcoin ÷ GitHub 19:06:07 <dongcarl> hi 19:06:08 <wumpus> jonasschnelli: ok, removed 19:06:20 <jonatack> jamesob: fjahr: reviewing your hi-prios atm 19:06:21 <wumpus> I mean jonatack* 19:06:42 <sdaftuar> #14053 -- is that still chasing concept ACK? i haven't followed the discussion but i htink it's been in that state for a long time 19:06:46 <gribble> https://github.com/bitcoin/bitcoin/issues/14053 | Add address-based index (attempt 4?) by marcinja ÷ Pull Request #14053 ÷ bitcoin/bitcoin ÷ GitHub 19:07:23 <wumpus> I think we can drop that 19:09:41 <wumpus> there's no discussion there by any actual contributors to the project 19:09:55 <sdaftuar> it seems it had many concept acks early on (2 years ago) 19:10:10 <fjahr> it has a lot of interest from the wider community 19:10:39 <jonatack> iirc it's desired by the community but unsustainable long-term (haven't looked in a long time but we did a review club on it) 19:10:41 <wumpus> I still think an address index over the entire block chain is bad idea and becomes a worse idea the more the thing grows, software that relies on that is broken 19:11:05 <sdaftuar> i thought it was just an index on the utxo set. it's indexing the whole blockchain? 19:11:10 <sipa> yes 19:11:26 <sdaftuar> ohh i see 19:11:28 <wumpus> it's the whole block chain I think? utxo index would make more sense to me 19:11:32 <fjahr> If people are against it, they should NACK it 19:11:47 <jonasschnelli> the eralier attemps where just for the utxo set, right? 19:11:59 <wumpus> fjahr: I have nacked similar things for years, if people keep digigng it up I lose the energy for that 19:12:03 <jonasschnelli> a complete index seems out of scope IMO 19:13:14 <jonatack> i soft-nacked it https://github.com/bitcoin/bitcoin/pull/14053#pullrequestreview-340041837 19:13:19 <wumpus> yes, there have been attemps for indexes over only the UTXO set, but they didn't make it in either for some reason (dunno why, maybe the author just gave up) 19:13:37 <sipa> it's a difficult question... (IMO) people shouldn't be building infrastructure that relies on having such an index, but if they're going to do it anyway, it's perhaps not any worse to have in bitcoin core 19:13:41 <MarcoFalke> It is no worse than -txindex 19:13:49 <sipa> txindex is terrible too 19:13:54 <sipa> but we can't just drop it 19:14:00 <wumpus> yes, it's just as worse as txindex 19:14:07 <fjahr> wumpus: understandable, maybe there should be some kind of "permanent concept NACK" if enough people agree so nobody puts effort into putting into core anymore, maybe there is a way there for a project that works closely with core 19:14:27 <aj> isn't that just a good reason for both addrindex and txindex to be optional and default off? 19:14:40 <wumpus> fjahr: I mean I have no interest in blocking something if people really want it, but I'm not going to be part of it 19:14:54 <wumpus> fjahr: and apparently a lot of contributors think like that 19:14:57 <jonasschnelli> #20664 is something that allows (very) slow address scanning at a fraction of disk space 19:14:58 <gribble> https://github.com/bitcoin/bitcoin/issues/20664 | Add scanblockfilters RPC call by jonasschnelli ÷ Pull Request #20664 ÷ bitcoin/bitcoin ÷ GitHub 19:15:04 <jonasschnelli> (with blockfilters) 19:15:26 <sipa> aj: yeah, maybe 19:15:41 <MarcoFalke> I won't use addrindex myself, but the whole point of ./src/indexes/ was to make it easier to add (crappy) indexes 19:15:43 <sipa> it's not like our resistance (or reluctance) to add something like that has stopped people from using it 19:16:19 <jonasschnelli> Sure. But there are dedicated projects like Electrs doing it on a pretty good level 19:16:27 <sipa> yes 19:16:31 <jonasschnelli> Which is basically project modularization 19:16:39 <jonasschnelli> They can use their optimized DB (rocksDB in that case) 19:16:43 <wumpus> yeah https://github.com/bitcoin/bitcoin/pull/14053#issuecomment-744015971 19:16:52 <sipa> MarcoFalke does have a point 19:17:05 <jonasschnelli> If we are going to merge it, people will build on top of it and maintenance/traffic goes into our project 19:17:06 <wumpus> no need to compete with dedicated projects whose whole point is to optimize this 19:17:15 <wumpus> bitcoin core is not a blob that needs to absorb the entire bitcoin ecosystem 19:17:23 <MarcoFalke> It would be nice if indexes could be attached to bitcoin core at runtime (external modules), but we are not there yet 19:17:35 <wumpus> jonasschnelli: indeed 19:17:35 <jonasschnelli> Adding a complete address index is against the project modularization. 19:17:43 <wumpus> it's scope creep 19:17:49 <jonasschnelli> I think adding more interfaces to allow "better" interaction would be the way to go 19:17:52 <sipa> at least the argument of it being invasive validation code changes doesn't apply anymore 19:17:53 <wumpus> and there are already solutions for it 19:18:19 <MarcoFalke> bitcoin-tx is scope creep as well (playing devils advocate) 19:18:22 <wumpus> MarcoFalke: well, external programs exist for it, it's supposedly possible 19:18:32 <jonasschnelli> I also see (and feel) the point that it would be "handy" to have it "in one box". But long term, it might a bad idea 19:18:48 <MarcoFalke> https://github.com/bitcoin-core/btcdeb is also scope creep 19:19:00 <wumpus> well then people could make a bitcoin core + electrs bundle, for example 19:19:24 <wumpus> there's lots of systems like docker that allow shipping multiple software that works together in one box 19:19:32 <sipa> my primary objection is "nobody should be using this" 19:20:00 <wumpus> MarcoFalke: that's literally a separate repository though :) 19:20:06 <sipa> but if people are going to use this anyway, and it can be added without being too invasive, and being optional... from a code maintenance perspective i don't have that much problems with it 19:20:17 <jonasschnelli> I guess there are reasons to use it,... if your backend servers hundreds of wallets and if you want instance seed-backup restores 19:20:26 <MarcoFalke> It is still a repo in our org ;) 19:20:27 <jonasschnelli> *instant 19:20:36 <wumpus> MarcoFalke: I don't have a problem with it being in the org 19:20:45 <jonasschnelli> Yes. 19:20:49 <wumpus> just not everything needs to be part of the bitcoin core repostiory 19:21:01 <jonasschnelli> The address index could be outside our repo and I think it would be fine 19:21:26 <jonasschnelli> Maybe similar like the guy... but ideally process separated from the beginning. 19:21:28 <jonasschnelli> *GUI 19:21:46 <wumpus> let's fork electrs into bitcoin-core then :) 19:21:55 <jonasschnelli> :/ 19:22:38 <aj> txindex is slightly helpful for speeding up our own RPCs (getrawtransaction), i don't think there's a similar rationale for needing an integrated addrindex? 19:23:35 <sipa> getrawtransactionsbyaddress, an RPC that very unhelpfully always reports the "RPC does not exist" error right now 19:24:19 <MarcoFalke> how does txindex speed things up? 19:24:22 <jonasschnelli> why is an index for the utxo set not enoght? 19:24:41 <jonasschnelli> If you want the spent history, use blockfilters 19:26:34 <sipa> i commented 19:27:01 <bitcoin-git> [bitcoin] sdaftuar opened pull request #20690: Clean up logging of outbound connection type (master...2020-12-clean-up-outbound-logging) https://github.com/bitcoin/bitcoin/pull/20690 19:27:08 <aj> MarcoFalke: compared to looping over every historical block and checking blockfilters, it's O(log(txs)) vs O(blocks) 19:27:14 <wumpus> it will just be the same discussion again that's been done many times over the years i expect 19:28:25 <jamesob> jonatack: thanks! 19:28:38 <jamesob> I have some feedback from wumpus to address I think 19:28:46 <fjahr> jonasschnelli: I think such a combo is worth exploring 19:29:12 <jonasschnelli> fjahr: look at #20664 (for performance analysis) 19:29:14 <gribble> https://github.com/bitcoin/bitcoin/issues/20664 | Add scanblockfilters RPC call by jonasschnelli ÷ Pull Request #20664 ÷ bitcoin/bitcoin ÷ GitHub 19:29:33 <sipa> so when rc4 topic? 19:29:47 <fjahr> jonasschnelli: will do 19:30:42 <wumpus> #topic rc4 status, macos codesigning fix (wumpus) 19:30:42 <core-meetingbot> topic: rc4 status, macos codesigning fix (wumpus) 19:31:13 <wumpus> I saw there are two parellel workarounds for the rc3 macos signing issue 19:31:23 <sipa> so we have two possible (temporary?) fixes for macos codesigning, #20638 and #20644 19:31:25 <gribble> https://github.com/bitcoin/bitcoin/issues/20638 | build: Fix macOS code signing by pre-allocating space for the code signature during gitian build by achow101 ÷ Pull Request #20638 ÷ bitcoin/bitcoin ÷ GitHub 19:31:27 <gribble> https://github.com/bitcoin/bitcoin/issues/20644 | Add patch to make codesign_allocate compatible with Apples by sipa ÷ Pull Request #20644 ÷ bitcoin/bitcoin ÷ GitHub 19:31:27 <wumpus> we'll probably want to choose one 19:31:32 <sipa> i think both are fine 19:31:56 <sipa> and if achow101's python codesigner works, all of this will be obsolete 19:32:04 <sipa> s/if/when/ - no pressure 19:32:11 <jonasschnelli> heh 19:32:14 <wumpus> but that's more of a long-run solution 19:32:17 <achow101> it's coming along slowly 19:32:35 <jonasschnelli> achow101: no pressure,.. but will it be ready for an rc4? :) 19:32:52 <achow101> I might have a PoC tomorrow 19:33:05 <jonasschnelli> Maybe for 0.21 we should use the existing solutions and use 0.22 (+backport) for achow101 signing tool 19:33:18 <wumpus> I don't think it's a good idea to introduce something like that between rcs 19:33:18 <sipa> yeah, not really relevant in this discussion; we can just pick one; my PR makes our codesign compatible with Apple's (to the extent that we know, but it can always change); achow's works around the issue (by predicting some things the Apple code will do) 19:33:21 <sipa> jonasschnelli: seems reasonable 19:33:42 <wumpus> unless there is no other workable option, of course 19:33:43 <jonasschnelli> sipa's PR is an easier fix and maybe less risky? 19:33:51 <jonasschnelli> (if we going to drop it anyways) 19:34:08 <achow101> I think sipa's PR potentially has other side effects that we haven't discovered 19:34:21 <sipa> well, the signature validates 19:34:23 <achow101> since it modifies cctools and we use more than just codesign_allocate in it 19:34:32 <jonasschnelli> sipa: it only needs to work for rc4 and final release 19:34:34 <sipa> hmm, do we? 19:34:41 <jonasschnelli> (oh... and maybe for 0.20.2) 19:34:47 <sipa> what else from cctools do we use? 19:35:55 <achow101> pagestuff, but it doesn't seem to be effected 19:35:56 <dongcarl> I believe we use it as our binutils 19:36:03 <dongcarl> For macOS targets 19:36:03 <sipa> the changed function only affects codesign_allocate, lipo, bitcode_strip, and segedit 19:36:06 <achow101> I thought we used some of the tools for building but I'm not sure 19:36:40 <sipa> anyway, no strong feelings 19:36:40 <dongcarl> I don't think we use any of the other ones sipa just listed, need to double check tho 19:36:43 <sipa> and no point spending much time on this 19:37:19 <achow101> well, at least during testing we didn't find any other problems, so it's probably fine 19:39:37 <wumpus> ok 19:39:56 <jonasschnelli> I propose to merge #20644 and do rc4 and 0.20.2rc1. Then wait for achow101 signing tool to have a stable long term fix 19:39:58 <gribble> https://github.com/bitcoin/bitcoin/issues/20644 | Add patch to make codesign_allocate compatible with Apples by sipa ÷ Pull Request #20644 ÷ bitcoin/bitcoin ÷ GitHub 19:40:08 <jonasschnelli> (then revert 20644) 19:40:14 <wumpus> so let's try merging sipa's patch and do rc4, if that runs into unexpected issues, try the other one? 19:40:22 <jonasschnelli> sure 19:40:26 <achow101> ack 19:40:27 <jonasschnelli> ack 19:40:32 <sipa> also #20646 in rc4 or not? 19:40:34 <gribble> https://github.com/bitcoin/bitcoin/issues/20646 | p2p: ignore post-verack sendaddrv2 instead of disconnecting by jonatack ÷ Pull Request #20646 ÷ bitcoin/bitcoin ÷ GitHub 19:40:44 <sipa> as pointed out, it makes little sense if it doesn't go in 0.21 19:40:55 <wumpus> sipa: I don't think people even agree about doing it 19:42:05 <sipa> yeah 19:42:12 <aj> it has three acks and a +/- 0? 19:42:51 <wumpus> my ACK is more of a 'I don't disagree with doing this and the code change looks correct' 19:44:29 <luke-jr> . 19:45:37 <MarcoFalke> ... 19:45:53 <aj> i don't think i have anything to add that's not already in the pr 19:46:03 <wumpus> same 19:46:13 <MarcoFalke> someone should flip a coin 19:46:31 <wumpus> if we're not sure it's better to err on the side of not making a code change 19:47:35 <aj> Jackielove4u spend 2h the other night debugging test failures because of it, likewise it broke syncing signet, i don't really see why this is a coin toss 19:47:41 <wumpus> especially if it's a merge-and-rush-into-rc4-last-minute thing 19:48:38 <sipa> aj: ah, other report of someone hit by it? 19:49:07 <aj> sipa: had bitcoin-qt and bitcoind refusing to connect, didn't realise bitcoin-qt was still rc2 19:49:17 <jonatack> i'm ambivalent, but it seems safer to ignore instead of disconnect if we're in doubt. the signet thing, and also a year ago we had to pull a release 19:49:55 <jonatack> and anything else we haven't hit related to it 19:50:21 <wumpus> 0.21.0 has had many of those 'we need to decide this last minute' and it doesn't make me comfortable 19:50:25 <MarcoFalke> The signet sync is already fixed, so merging the pull won't change that 19:50:33 <jonatack> wumpus: i empathize 19:50:52 <aj> wumpus: there were a lot of big merges just before freeze 19:51:30 <wumpus> MarcoFalke: great! 19:51:45 <luke-jr> maybe we need more time between freeze and rc1? (though I dislike the duration) 19:52:00 <luke-jr> perhaps branch at freeze and tag rc1 weeks afterward? 19:52:12 <wumpus> that's just more backporting work 19:52:20 <MarcoFalke> luke-jr: That'd mean too much work backporting 19:52:23 <jonasschnelli> luke-jr: most stuff appears in rcs.. :/ 19:52:45 <MarcoFalke> Also, most rc issues are build system issues 19:52:57 <wumpus> jonasschnelli: bugs appearing and fixed in rcs is completely normal and expected, what bothers me is these kind of last minute protocol handling changes 19:53:05 <MarcoFalke> So you won't find them before the tag 19:53:09 <jonasschnelli> wumpus: indeed. 19:53:19 <jonasschnelli> Although, ignoring them is also no option 19:53:38 <wumpus> jonasschnelli: well if MarcoFalke says the signet sync issue is fixed 19:53:41 <jonasschnelli> MarcoFalke: good point 19:53:56 <sipa> well it's fixed for people who update 19:54:03 <sipa> no? 19:54:09 <sipa> that was already the case 19:54:37 <aj> the original issue was people running rc3 couldn't sync from the non-tor seed node 19:54:42 <MarcoFalke> sipa: The seed node was on rc2, so a user updating to rc3 *broke* it 19:54:53 <sipa> ah i see 19:54:59 <sipa> ok, fair point 19:55:07 <MarcoFalke> *pre-rc2 (not sure if it was actually rc2) 19:55:08 <aj> that node is now updated, so now anyone running rc2 and earlier can't sync from it instead 19:55:29 <MarcoFalke> aj: Correct 19:55:43 <aj> i think the seed node was a previous version of #19937 19:55:46 <gribble> https://github.com/bitcoin/bitcoin/issues/19937 | signet mining utility by ajtowns ÷ Pull Request #19937 ÷ bitcoin/bitcoin ÷ GitHub 19:55:48 <sipa> so it's not really signet-specific anymore... the question is just whether gratuitous disconnection of existing old nodes is worth slightly better observability of people implementing the protocol incorrectly 19:55:54 <MarcoFalke> aj: Though, that seems like a feature (telling people to upgrade from non-supported versions) 19:56:20 <wumpus> there isn't any released node that will be disconnected right? 19:56:20 <aj> MarcoFalke: telling people to upgrade sure, but this is just "it fails for no apparent reason" 19:56:29 <sipa> wumpus: correct 19:57:07 <wumpus> I mean if you implement the ignoring someone might spent hours debugging why addrv2 doesn't work 19:57:10 <sipa> it was affecting signet disproportionally because of course all signet nodes were running pre-release code 19:57:24 <wumpus> sometimes I wish we still had the reject message 19:57:47 <MarcoFalke> the reject message was replaced by -debug=net 19:58:01 <wumpus> it could send an error to the peer before disconnecting, or in general when not accepting it 19:58:03 <sipa> wumpus: maybe, but if they're trying to make it work is "why am i getting addr and not addrv2?" really that different from "why am i being disconnected?" 19:58:14 <aj> it would have been nice if there was this much resistance to #20564 :-/ 19:58:18 <gribble> https://github.com/bitcoin/bitcoin/issues/20564 | Dont send sendaddrv2 to pre-70016 software, and send before verack by sipa ÷ Pull Request #20564 ÷ bitcoin/bitcoin ÷ GitHub 19:58:52 <sipa> aj: later and later in the process, more and more resistance... 19:58:54 <wumpus> aj: that was also a reluctant ACK for me 19:58:57 <luke-jr> aj: test != mainnet 20:00:07 <wumpus> but that was about mainnet software, and actual releases of alternative clients 20:00:14 <aj> sipa: pretty easy to have a BCLog::NET debug message of either "Ignoring SENDADDRV2 received after VERACK" or "SENDADDRV2 received after VERACK; disconnecting" -- we have neither of those atm, also for WTXIRELAY 20:00:36 <sipa> aj: yeah, that would be a good idea 20:00:44 <wumpus> yes, would make sense to add that 20:01:27 <aj> i have a patch to add it, but was waiting for the sendaddrv2 to get fixed (or, i guess, nacked) 20:02:10 <wumpus> it's time to end the meeting 20:02:11 <jonatack> disconnection was added pretty late in the 20564 review process 20:02:29 <aj> luke-jr: i guess i don't really agree. the whole point of a test net is to catch breakage first; if you catch breakage on the testnet and then go "oh, it's just broken on the testnet, it'll be fine on mainnet" that kind of defeats the purpose imo 20:02:32 <jonatack> (i didn't disagree with it, just observing) 20:03:03 <wumpus> #endmeeting