19:00:49 <wumpus> #startmeeting 19:00:50 <core-meetingbot> Meeting started Thu Nov 26 19:00:49 2020 UTC. The chair is wumpus. Information about MeetBot at https://bitcoin.jonasschnelli.ch/ircmeetings. 19:00:50 <core-meetingbot> Available commands: action commands idea info link nick 19:00:53 <sipa> hebasto: yes, but what when we introduce v3? 19:01:00 <jonasschnelli> hi 19:01:06 <wumpus> #bitcoin -core-dev Meeting: achow101 aj amiti ariard bluematt cfields Chris_Stewart_5 digi_james dongcarl elichai2 emilengler fanquake fjahr gleb gmaxwell gwillen hebasto instagibbs jamesob jb55 jeremyrubin jl2012 jnewbery jonasschnelli jonatack jtimon kallewoof kanzure kvaciral lightlike luke-jr maaku marcofalke meshcollider michagogo moneyball morcos nehan NicolasDorier paveljanik 19:01:06 <kanzure> hi 19:01:07 <hebasto> hi 19:01:07 <wumpus> petertodd phantomcircuit promag provoostenator ryanofsky sdaftuar sipa vasild wumpus 19:01:09 <achow101> today's a us holiday so there may be fewer people 19:01:21 <jonatack> hola 19:01:34 <glozow> hi 19:01:41 <wumpus> congrats on rc2 everyone ! 19:01:44 <fjahr> hi 19:01:45 <jb55> hi 19:01:54 <wumpus> achow101: yes, might be a short meeting 19:01:54 <sipa> hi 19:02:11 <michaelfolkson> Happy Thanksgiving US peeps 19:02:11 <ajonas> hi 19:02:50 <wumpus> we do have two proposed meeting topics for today: package validation design question (glozow), 2019-20 Coredev survey summary (ajonas) 19:03:18 <wumpus> any any others if people have last minute proposals 19:03:46 <wumpus> #topic High priority for review 19:03:46 <core-meetingbot> topic: High priority for review 19:04:03 <wumpus> https://github.com/bitcoin/bitcoin/projects/8 9 blockers, 2 chasing concept ACK 19:04:32 <jnewbery> hi 19:04:43 <wumpus> anything to add/remote or that is ready for merge? 19:04:56 <fjahr> Can we add #19055 to blockers again? 19:05:00 <gribble> https://github.com/bitcoin/bitcoin/issues/19055 | Add MuHash3072 implementation by fjahr ÷ Pull Request #19055 ÷ bitcoin/bitcoin ÷ GitHub 19:05:34 <wumpus> fjahr:sure 19:05:41 <fjahr> thx 19:05:58 <sipa> #20207 should be ready... not really a blocker for me, but would be nice to get in 19:06:00 <gribble> https://github.com/bitcoin/bitcoin/issues/20207 | Follow-up extra comments on taproot code and tests by sipa ÷ Pull Request #20207 ÷ bitcoin/bitcoin ÷ GitHub 19:06:36 <wumpus> you have nothing else on the list so will add it 19:08:04 <wumpus> that concludes the topic I think 19:08:25 <jonatack> not as a blocker, but maybe #20483 for backport to 0.21 to avoid the feeRate / fee_rate options being a potential source of confusion/footgun 19:08:27 <gribble> https://github.com/bitcoin/bitcoin/issues/20483 | wallet: deprecate feeRate in fundrawtransaction/walletcreatefundedpsbt by jonatack ÷ Pull Request #20483 ÷ bitcoin/bitcoin ÷ GitHub 19:08:45 <wumpus> ok 19:08:45 <jonatack> if not, otherwise it can wait 19:09:33 <wumpus> for 0.21.1 I guess? if it's not a bugfix I don't think we should merge it between rcs 19:09:40 <jonatack> (feeRate is in BTC/kB, fee_rate is in sat/vB) 19:10:04 <jonatack> wumpus: as you think best 19:10:09 <wumpus> in any case it's in the high prio list now 19:11:22 <jonatack> ð 19:11:43 <wumpus> oh it's 8 files changes, of which 7 tests and only small changes in a cpp 19:12:39 <wumpus> could still be in a rc i guess 19:13:09 <wumpus> #topic package validation design question (glozow) 19:13:09 <core-meetingbot> topic: package validation design question (glozow) 19:13:26 <glozow> So IâÂÂm in the process of doing package mempool acceptance logic. I have a design question that boils down to âÂÂif a transaction in a package fails, should we reject the whole package immediately?â 19:13:37 <glozow> If itâÂÂs a package from p2p (i.e. through package relay) then it seems most logical to fail as early as possible. my draft implementation does all PreChecks first for this reason, and then we donâÂÂt do script checks. 19:13:42 <glozow> If weâÂÂre doing a testmempoolaccept for a package, however, it seems appropriate to fully validate each one so clients have more helpful information. 19:14:26 <glozow> Wondering if people have thoughts/opinions? 19:14:48 <sipa> you're basically asking if the whole package would fail, should the behavior sort of automatically retry with subpackages? 19:15:03 <sipa> (or be equivalent to that) 19:15:14 <glozow> sipa: yeah 19:15:34 <sipa> is there a need for that? 19:15:46 <jnewbery> That sounds reasonable to me. package acceptance over p2p should be atomic. If you're doing a testmempoolaccept it's helpful to return more granular acceptance information 19:16:06 <sipa> the idea is initially just having this as RPC? 19:16:14 <wumpus> yes, P2P clients don't get helpful information back anyway 19:16:16 <jnewbery> If there was a sendrawtransactionpackage we'd probably want that to be atomic I think 19:16:33 <glozow> yeah, i want to do 1 testmempoolaccept, 2 submit packages through rpc, 3 package relay 19:16:51 <sipa> no reason why the RPC can't be initially just only entire-package, and if there is a use for more granular acceptance, add RPC support for that later 19:16:55 <sipa> (if ever) 19:17:09 <wumpus> for RPC it's useful to at least get detailed information on which transaction failed, but it still makes sense for it to be atomic 19:17:50 <sipa> i could imagine that on P2P maybe there are reasons why you'd want to be able to accept subpackages, e.g. to avoid one bad transactions added to a package by an attacker can't prevent the entire package from being relayed 19:17:56 <sipa> but i think those are longer term questions 19:17:58 <jnewbery> I think if you submit a package (tx A -> tx B -> tx C) to testmempoolaccept it's helpful to the user to know whether A or B or C failed 19:18:42 <glozow> yes, i'm leaning towards running full checks for each tx until they pass/fail in a testaccept 19:19:16 <michaelfolkson> But surely something has gone badly wrong if you can't figure out which transaction in your package was the reason for the rejection? 19:19:16 <wumpus> sipa: right, shouldn't cache all the transactions individually as rejected 19:19:51 <glozow> yah good point 19:20:32 <michaelfolkson> I guess it depends on whether if there are weird rejection policies out there 19:21:00 <glozow> michaelfolkson: what kinds of weird rejection policies? 19:21:03 <jnewbery> Do we ever need to sort the txs in a package (from user or over P2P), or do we expect them always to be sorted? 19:21:17 <glozow> jnewbery: i think we should sort it ourselves 19:21:18 <sipa> jnewbery: i'd expect to use dependency ordering 19:21:40 <sipa> (first sort by how many parents in the package it has, then by txid) 19:21:51 <sipa> on either the sender or the receiver side 19:22:00 <michaelfolkson> glozow: I'm trying to think of reasons for a package rejection that isn't clear to the sender 19:22:35 <sipa> michaelfolkson: the obvious one is a new standardness rule on the network (e.g. softfork that enables new scripts, which the sender knows about but the receiver doesn't) 19:22:42 <sipa> if that happens in a leaf of a package 19:22:48 <jnewbery> sipa: how many parents or how many ancestors? 19:23:02 <sipa> jnewbery: either works 19:23:17 <sipa> presumably the receiver still wants the package without that leaf 19:23:17 <glozow> would you know how many ancestors without doing a few checks? uh oh 19:23:32 <sipa> glozow: that's trivial, i think? 19:23:38 <sipa> we do that sorting everywhere 19:23:41 <sipa> inside tx messages e.g. 19:23:45 <jnewbery> huh? A tx's parent can have more parents in the package than the child, no? 19:23:46 <glozow> ah okay cool 19:24:19 <sipa> jnewbery: oh sorry i mean you can either use ancestors, or depth 19:24:23 <sipa> not # of parents 19:24:28 <jnewbery> right 19:24:40 <sipa> but i think these are all questions for later 19:24:47 <jonatack> glozow: is your concern about a trade-off between resource use/ddos and full atomic validation? 19:25:01 <sipa> for now i think, as an RPC it's fine to just try atomically accepting the whole package 19:25:08 <michaelfolkson> sipa: Is it possible that post Taproot activation that an unupgraded node would reject Taproot transactions in a package? 19:25:19 <sipa> michaelfolkson: they should 19:25:31 <sipa> (except there won't be pre-taproot post-package relay nodes :p) 19:26:16 <michaelfolkson> sipa hopes 19:26:18 <michaelfolkson> haha 19:26:35 <sipa> regardless of whether it activates or not 19:26:46 <glozow> jonatack: yeah, even if not a dos concern, it at least seems unproductive to run PolicyScriptChecks for any tx if we're going to reject the whole package, since those aren't even cached. 19:27:01 <sipa> policy checks are cheap though 19:27:15 <glozow> PolicyScriptChecks? 19:27:33 <jnewbery> sipa: PolicyScriptChecks are script/sig validation 19:27:39 <sipa> oh, ok 19:27:44 <sipa> not familiar with that function 19:28:05 <michaelfolkson> Why "they should"? What do they have to lose by accepting a anyonecanspend transaction as part of the package? 19:28:26 <sipa> michaelfolkson: soft fork safety relies on using transactions the unupgraded networks considers nonstandard 19:28:43 <jnewbery> sipa: it's a step in AcceptToMemoryPool 19:29:18 <jnewbery> added by sdaftuar when he split it possible to implment package acceptance I believe 19:29:30 <sipa> ah 19:29:51 <glozow> yeah added in #16400 19:29:57 <gribble> https://github.com/bitcoin/bitcoin/issues/16400 | refactor: Rewrite AcceptToMemoryPoolWorker() using smaller parts by sdaftuar ÷ Pull Request #16400 ÷ bitcoin/bitcoin ÷ GitHub 19:30:35 <jnewbery> sipa: so you think the node should always dependency sort packages, both from p2p and rpc? 19:30:53 <jnewbery> seems to me that the sender/client should do that? 19:31:06 <sipa> jnewbery: yeah, no opening on whose responsibility it should be 19:31:23 <sipa> in P2P, i think it may make sense to force the burden on the sender, because they already know this information anyway 19:32:15 <glozow> sipa: so if you receive a package you don't check to see if it's sorted properly, you just reject if it doesn't work? 19:32:38 <jnewbery> I can see the argument for rpc to accept just a bag of txs and sort it 19:32:44 <sipa> glozow: yeah 19:32:46 <sipa> jnewbery: yeah 19:32:49 <glozow> (also so i don't take up too much time, just making sure, for RPC, it's ok if i do full validation checks for each tx in a testmempoolaccept package, and atomic for real package submissions?) 19:33:13 <sipa> glozow: i'd more think the other way around 19:33:22 <wumpus> there's one other, probably short, topic, so no you're not taking up too much time :) 19:33:29 <jnewbery> I think sipa is saying make both RPC and P2P atomic 19:33:42 <sipa> i'm saying, for now, only do RPC, and make it atomic 19:33:51 <sipa> because there is no need for anything else unless someone comes up with a use case 19:34:11 <sipa> for P2P package relay... i feel there may be a need for accepting subpackages, but that's a question we can discuss later 19:34:18 <jnewbery> I still think there's value to make RPC granular if it doesn't add too much complication, because providing the extra information to the user can be helpful 19:34:35 <sipa> extra information and being atomic aren't in conflict with each other 19:34:50 <sipa> you can report "the package failed... and this is the first problem i encountered, in tx A" 19:34:59 <wumpus> right 19:35:23 <jnewbery> ah, maybe we haven't described the problem well 19:35:36 <sipa> being non-atomic would be a result of the form "the subset of packages {A,B,C,...} is acceptable from your bag, but D failed because Y" 19:35:51 <jnewbery> there are many steps in AcceptToMemoryPool: Prechecks, policy checks, consensus checks 19:35:56 <sipa> at least that's how i interpreted it 19:36:18 <michaelfolkson> Non-atomic would be accepting a subset of the package right? Rather than rejecting it outright and providing info why 19:36:22 <jnewbery> we should first do prechecks for all txs, then policy checks for all txs, then consensus checks for all txs 19:36:35 <sipa> jnewbery: agree 19:36:59 <glozow> so the question is, if A and B passed prechecks but C didn't, should we still run script checks for A and B? 19:37:11 <jnewbery> but if tx C fails prechecks, it might still be helpful to do policy checks for tx A and tx B and consensus checks for tx A and tx B so we can say to the user "tx A and tx B are good, but tx C fails in prechecks" 19:37:34 <sipa> i'd say no - for now - because the package in its entirety isn't acceptable 19:38:23 <michaelfolkson> I guess the concern that there is inefficiency. The sender keeps coming back with a smaller package that still gets rejected 19:38:43 <sipa> michaelfolkson: in RPC there is no "sender" 19:38:59 <glozow> what if there's a script error in A, and a prechecks error in C (and C depends on B which depends on A), it'd be more helpful to the RPC client to know how to fix the error in A first? 19:39:01 <sipa> and if the user wants that kind of functionality, we should add it 19:39:36 <sipa> glozow: is that a big design question that needs to be known up front? 19:39:46 <sipa> otherwise i'd just say do the simplest thing first, and iterate 19:41:59 <glozow> it does affect the testmempoolaccept API, i'd like as helpful a response as possible if the package fails 19:42:27 <sipa> i feel that adding more detailed error reporting later can probably be done in a backward compatible manner 19:42:41 <MarcoFalke> I'd also say to do the simpler thing first. Literally any progress on teaching TATMP packages is a nice feature 19:42:53 <glozow> ok, so atomic for now 19:42:56 <sipa> if the RPC is "your package is acceptable in its entirety: yes/no. here is a list of problems i found: []" 19:43:10 <MarcoFalke> Also, we can't keep policy error messages identical between releases anyway 19:43:17 <sipa> MarcoFalke: right 19:44:07 <glozow> got it, thanks everyone 19:44:37 <wumpus> #topic 2019-20 Coredev survey summary (ajonas) 19:44:37 <core-meetingbot> topic: 2019-20 Coredev survey summary (ajonas) 19:44:48 <MarcoFalke> glozow: Thanks for working on this! 19:44:56 <sipa> absolutely 19:45:02 <ajonas> In January, jnewbery sent out a survey that he planned to present the results of at the Coredev in March. Given that the meeting never happened, I put together a presentation/post summarizing the responses. Both can be found at https://adamjonas.com/bitcoin/coredev/retro/coredev-2019-retro/. 19:45:14 <ajonas> No action items. Just want to call it to people's attention. 19:45:22 <wumpus> ajonas: awesome! 19:45:46 <jnewbery> ajonas: thank you for picking this up 19:45:55 <michaelfolkson> For the summary watch the video I guess? 19:46:12 <ajonas> the post and the video are about the same 19:46:15 <sipa> short note: i just opened #20511, which i think is something to address for 0.21 19:46:16 <gribble> https://github.com/bitcoin/bitcoin/issues/20511 | anchors.dat doesnt support V2 addresses ÷ Issue #20511 ÷ bitcoin/bitcoin ÷ GitHub 19:46:20 <jonatack> ajonas: wow! thanks! 19:46:26 <glozow> ajonas: wow cool!!! 19:46:34 <sipa> ajonas: thanks for that, will read (and maybe watch) 19:46:43 <jonasschnelli> nice! 19:46:44 <ajonas> thanks to jnewbery for sending it out and I hope we can do another round in early 2021. 19:47:46 <fjahr> ajonas: cool! 19:49:41 <wumpus> unless anyone wants to discuss a specific thing from that overview, I think this concludes the meeting 19:49:42 <jnewbery> I agree that there's much more value in these things if we repeat them periodically. Everyone who answered talked about their hopes for the project and their contributions in 2020, so looking back at those should be interesting for people 19:50:51 <wumpus> yes it's interesting I agree with most of the comments 19:51:25 <wumpus> #endmeeting