14:00:11 <achow101> #startmeeting 14:00:15 <TheCharlatan> hi 14:00:24 <fjahr> hi 14:00:27 <achow101> #bitcoin -core-dev Meeting: achow101 _aj_ amiti ariard aureleoules b10c BlueMatt brunoerg cfields darosior dergoegge dongcarl fanquake fjahr furszy gleb glozow hebasto instagibbs jamesob jarolrod jonatack josibake kallewoof kanzure kouloumos kvaciral laanwj LarryRuane lightlike luke-jr MacroFake Murch phantomcircuit pinheadmz promag provoostenator ryanofsky sdaftuar S3RK stickies-v sipa theStack TheCharlatan vasild 14:00:35 <hebasto> hi 14:00:36 <brunoerg> hi 14:00:38 <b10c> hi 14:00:44 <darosior> hi 14:00:46 <pinheadmz> hi where r u, ,meeting bot ? 14:00:49 <achow101> There are no pre-proposed meeting topics. Any last minute topics to add to the list? 14:01:04 <dergoegge> hi 14:01:35 <lightlike> hi 14:01:37 <stickies-v> hi 14:02:11 <achow101> #topic assumeutxo updates (jamesob) 14:02:59 <instagibbs> looks like #27746 is the thing to look at 14:03:01 <gribble`> https://github.com/bitcoin/bitcoin/issues/27746 | Rework validation logic for assumeutxo by sdaftuar ÷ Pull Request #27746 ÷ bitcoin/bitcoin ÷ GitHub 14:03:01 <achow101> #27746 seems to be the current pr. it's been getting more review and activity 14:03:02 <gribble`> https://github.com/bitcoin/bitcoin/issues/27746 | Rework validation logic for assumeutxo by sdaftuar ÷ Pull Request #27746 ÷ bitcoin/bitcoin ÷ GitHub 14:03:26 <josie> hi 14:03:41 <sdaftuar> thank you reviewers! will address latest comments today 14:04:23 <Murch> Hi 14:04:55 <achow101> #topic package relay updates (glozow) 14:05:37 <glozow> thanks yâÂÂall for your reviews 14:06:34 <achow101> #26711 looks to be getting more review 14:06:37 <gribble`> https://github.com/bitcoin/bitcoin/issues/26711 | validate package transactions with their in-package ancestor sets by glozow ÷ Pull Request #26711 ÷ bitcoin/bitcoin ÷ GitHub 14:07:03 <glozow> yes! it seems we may want to brainstorm the approach a bit more 14:07:48 <glozow> thereâÂÂs a tradeoff for computational complexity⦠idk if we can handle all possible edge cases without eg trying every possible subset of txns 14:09:03 <instagibbs> https://github.com/bitcoin/bitcoin/pull/26711#discussion_r1229432536 <--- regarding this? 14:10:15 <glozow> yeah 14:10:19 <achow101> there's also https://github.com/bitcoin/bitcoin/pull/26711#discussion_r1226067022 14:11:12 <_aj_> (hi) 14:11:26 <fanquake> (gday) 14:11:29 <sipa> that sounds vaguely related to cluster linearization... 14:12:06 <_aj_> sipa: i thought so too 14:12:17 <instagibbs> Status quo means we're not considering subpackages at all, so I'm not sure I'm too worried about weird edge cases 14:12:39 <glozow> yeah. it also gets a bit more complicated if rbf is enabled⦠14:13:02 <Murch> All transactions are rbf-able, some are just in denial :p 14:13:10 <achow101> istm a simple chain would be O(n^2) PreChecks? which includes utxo lookup 14:13:27 <lightlike> will these problems be made trivial to handle after clustermempool? 14:13:36 <instagibbs> PreChecks are cheap, but can also be split out sure 14:13:40 <instagibbs> (or are they not) 14:14:08 <glozow> yeah. i mean there is a coinsview cache so you only load utxos once 14:14:33 <instagibbs> If we're not worried about prechecks, then the other concern is doing more work elsewhere than necessary, so tightening up when we retry is another thing we could try 14:14:37 <instagibbs> I think right now it's kind of broad? 14:14:50 <josie> Murch: miner deniers lol 14:15:24 <glozow> i dont think cluster mempool would make a difference here, as these txns arent in mempool yet. though perhaps there could be code/algos to reuse? 14:16:11 <instagibbs> linearization should be doing all the fee analysis bits though? 14:16:54 <sipa> We'll have code for "given a set of interconnected transactions, find a good linearization for it". Would that be helpful, as in the linearization gives you an order to try transactions in? (I haven't paid attention to package validation at all, this may be wildly off) 14:17:03 <sdaftuar> sorry i'm a bit behind on the conversation. the issue is whether we should evaluate a subpackage instead of a full package? 14:17:22 <sdaftuar> and if so, how do we find the subpackages? 14:17:40 <instagibbs> sipa "good linearization" in which terms, to clue me and others in 14:17:44 <sdaftuar> i believe this is equivalent to determining whether a package would be chunked together in the cluster-mempool model, which should be straightforward to do 14:18:25 <glozow> i think the issue is that this algo might not be handling all the edge cases we may be concerned about, and also itâÂÂs unclear if the computational complexity is ok. 14:18:38 <sipa> instagibbs: A linearization is always topologically valid (as in: parents always go before children). It's good if it moves as much of the feerate to the front of the linearization (e.g. respecting CPFP would make a linearization better). 14:19:55 <glozow> er, what do u do if an earlier tx in the package rbfs something, impacting a later tx in the package? ð 14:19:55 <sipa> If it's just chunking (you're already given the order that transaction go in in, but you wonder which ones should be considered together for CPFP like reasons), we have an O(n) algorithm. 14:20:54 <sdaftuar> glozow: my model for RBF right now is to simulate what the mempool would look like assuming all RBFs go through, sort the cluster of the RBF transaction (or package) to determine mining score of the new transaction (or package, i guess), and then compare against what would be evicted 14:21:07 <sdaftuar> i think that can all happen fast enough that there's not a CPU concern 14:21:55 <_aj_> yeah, fee calculations should be fast, so doing that for lots of different possible subpackages seems okay (as long as it's not 2**n different subpackages..) 14:22:33 <sdaftuar> _aj_: i believe in the cluster mempool world, we would just need to do a single cluster sort to get mining score for all transactions in a package. 14:22:40 <sdaftuar> and the chunks that are output would be the subpackages to consider 14:22:52 <glozow> so we could do something like: grab all utxos, linearize to find subpackages, validate assuming everything goes in, delete invalid ones, then do final fee analysis before submitting? 14:24:04 <sdaftuar> yes i think so. to expand a bit, "validate assuming everythign goes in" is just removing transactions from the cluster that the new package would be joining that are conflicted by the package. 14:24:32 <sdaftuar> i should say, tentatively removing transactions, from a putative new cluster that includes the new transactions 14:24:41 <sdaftuar> then we sort that, and look at the chunks. 14:24:49 <_aj_> sdaftuar: we need to actually check sigs on the new txs at some point; i think that's what glozow meant by validate here 14:24:57 <sdaftuar> ah, yes i think we would do that last 14:25:06 <sdaftuar> after checking feerates from the cluster sort 14:25:08 <glozow> also actually i change my previous answer, cluster mempool would eliminate concerns about submitting something that is handled poorly by the eviction algo 14:26:54 <glozow> what do we do if we find a bad sig? itâÂÂd impact the fee analysis, so would we linearize again and run rbf checks again after removing the invalid one(s)? 14:27:16 <instagibbs> post-linearization, I'd kind of expect it to just stop with failure? 14:27:20 <sipa> Is it a problem to in that case just treat the entire package as invalid? 14:27:40 <sdaftuar> sipa: not permanently invalid, but yes i don't think we need to find a valid subpackage 14:27:51 <sipa> Right. 14:27:56 <_aj_> could just leave the linearization, but drop any descendents of the invalid thing? 14:28:06 <instagibbs> ^ yeah 14:28:11 <glozow> yeah it should be ok as long as we are willing to try again with another peer 14:28:11 <sdaftuar> _aj_: that could admit non-feerate-passing transasctions into the mempool 14:28:39 <sdaftuar> seems simpler to give up, no peer should give us an invalid signature 14:28:55 <sdaftuar> so as long as we try from other peers, i think it's fine? 14:29:09 <_aj_> sdaftuar: i was treating the linearisation as giving us chunks which we'd validate as a set, i guess 14:29:35 <sdaftuar> _aj_: ok, i guess it woudl work if we required that chunks get accepted all-or-none 14:29:56 <sdaftuar> so if there's a package with two distinct chunks, we evaluate those separately and atomically 14:30:04 <_aj_> i can't see any reason not to just drop a package with a bad sig in it though 14:30:06 <glozow> is there linearization code available to use? i can mark 26711 as draft and rework it to do this 14:30:18 <_aj_> (and the peer that sent it) 14:30:19 <sdaftuar> glozow: uh i can share a branch. it needs a lot of work though 14:30:33 <_aj_> speaking of, review beg for #26291 :) 14:30:34 <sdaftuar> but it should show the idea for how this would work 14:30:35 <gribble`> https://github.com/bitcoin/bitcoin/issues/26291 | Update MANDATORY_SCRIPT_VERIFY_FLAGS by ajtowns ÷ Pull Request #26291 ÷ bitcoin/bitcoin ÷ GitHub 14:31:12 <achow101> sounds like we might get (part of) cluster mempool sooner than expected :) 14:31:28 <sdaftuar> glozow: also this only works with bounded clusters -- otherwise CPU DoS is not something we can deal with 14:31:48 <glozow> sdaftuar: thatâÂÂd be good! yeah maybe we can push bits of cluster mempool in :P 14:32:13 <sdaftuar> honestly i'm not sure how feasible that is but happy to get more eyes on it :) 14:32:18 <sipa> We're thinking up to 64 transactions can probably run in <0.1 ms on decent hardware (for ancestor-based sort, not optimal). 14:32:46 <glozow> oh yeah this came up in the bip as well. is 100 a realistic cluster limit? 14:32:57 <fanquake> what is the definition of "decent hardware" at the moment? 14:33:09 <sipa> fanquake: My Ryzen 5950X? 14:33:11 <glozow> (ie do you see cluster limit being bigger than 100?) 14:33:24 <sdaftuar> also my power9 machine i think 14:33:26 <fanquake> sipa: ok 14:33:30 <instagibbs> (or chunk limit) 14:33:43 <sipa> 100 seems high already 14:33:48 <glozow> asking because trying to come up with a good maximum count for getpkgtxns 14:34:14 <glozow> i thought so as well 14:34:56 <achow101> I think we're getting a little bit off topic, so I'm gonna cut y'all of there for time, but this is definitely something that should be continued after the meeting/offline. 14:35:08 <instagibbs> Draft it, we can chat in PR I guess 14:35:17 <glozow> sounds good thanks! 14:35:19 <achow101> #topic libbitcoinkernel updates (TheCharlatan) 14:35:49 <TheCharlatan> Since last week, #27576 got merged completing the gArgs removal in kernel code:) 14:35:51 <gribble`> https://github.com/bitcoin/bitcoin/issues/27576 | kernel: Remove args, settings, chainparams, chainparamsbase from kernel library by TheCharlatan ÷ Pull Request #27576 ÷ bitcoin/bitcoin ÷ GitHub 14:36:06 <TheCharlatan> The shutdown PR got split up a bit for ease of review, focus should currently be on #27866 and #27862. They are both small changes and self-contained. 14:36:07 <gribble`> https://github.com/bitcoin/bitcoin/issues/27866 | blockstorage: Return on fatal flush errors by TheCharlatan ÷ Pull Request #27866 ÷ bitcoin/bitcoin ÷ GitHub 14:36:07 <achow101> #27711 is currently draft and needs rebase, should something replace it as the primary blocker? 14:36:08 <gribble`> https://github.com/bitcoin/bitcoin/issues/27862 | validation: Stricter assumeutxo error handling when renaming chainstates by ryanofsky ÷ Pull Request #27862 ÷ bitcoin/bitcoin ÷ GitHub 14:36:10 <gribble`> https://github.com/bitcoin/bitcoin/issues/27711 | kernel: Remove shutdown globals from kernel library by TheCharlatan ÷ Pull Request #27711 ÷ bitcoin/bitcoin ÷ GitHub 14:37:07 <achow101> TheCharlatan: have a preference? 14:37:45 <TheCharlatan> no, I think both are equally important. 14:37:57 <ryanofsky> 27861 contains most of what was in 27711 and is good shape, already acked it 14:38:52 <achow101> put 27862 as primary 14:39:14 <TheCharlatan> good 14:40:02 <bitcoin-git> [bitcoin] fanquake pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/9372ec71e8a3...c454395115a8 14:40:02 <bitcoin-git> bitcoin/master 6779e6e dimitaracev: test: clean up is node stopped 14:40:02 <bitcoin-git> bitcoin/master c454395 fanquake: Merge bitcoin/bitcoin#27895: test: clean up is node stopped 14:40:08 <bitcoin-git> [bitcoin] fanquake merged pull request #27895: test: clean up is node stopped (master...test-cleanup-is-node-stopped) https://github.com/bitcoin/bitcoin/pull/27895 14:40:26 <achow101> #topic BIP 324 updates (sipa) 14:41:00 <sipa> Hi. Progress on the libsecp256k1 PR for ellswift! Once that's merged, I will look into opening more Bitcoin Core PR(s). 14:41:56 <achow101> cool 14:42:10 <achow101> #topic Ad-hoc high priority for review 14:42:28 <achow101> anything to add or remove from https://github.com/orgs/bitcoin/projects/1/views/4 14:43:00 <_aj_> #26291 since i'm begging, i guess? :) 14:43:01 <gribble`> https://github.com/bitcoin/bitcoin/issues/26291 | Update MANDATORY_SCRIPT_VERIFY_FLAGS by ajtowns ÷ Pull Request #26291 ÷ bitcoin/bitcoin ÷ GitHub 14:43:19 <achow101> _aj_: added 14:43:21 <Murch> #26152 is ready for review, would love to get some eyes on that 14:43:24 <gribble`> https://github.com/bitcoin/bitcoin/issues/26152 | Bump unconfirmed ancestor transactions to target feerate by Xekyo ÷ Pull Request #26152 ÷ bitcoin/bitcoin ÷ GitHub 14:44:17 <achow101> Murch: added 14:44:51 <achow101> Any other topics to discuss? 14:46:07 <_aj_> oh, 1 thing 14:46:39 <_aj_> on cirrus ci, i seem to only be able to rerun jobs for my own PRs; i thought i used to be able to trigger reruns for other people's prs as well. is that new, and if so, is it deliberate? 14:47:01 <brunoerg> Perhaps #26969 seems ready for merge I guess, but if more reviews is needed could it be added? seems good to move on to not cause rebase problems 14:47:03 <gribble`> https://github.com/bitcoin/bitcoin/issues/26969 | net, refactor: net_processing, add `ProcessCompactBlockTxns` by brunoerg ÷ Pull Request #26969 ÷ bitcoin/bitcoin ÷ GitHub 14:47:45 <achow101> _aj_: I don't think we've changed any permissions recently, could be a cirrus change. but iirc it was always the case that you could only re-run your own prs 14:48:21 <_aj_> achow101: entirely possible that i may be misremembering 14:48:22 <fanquake> brunoerg: unfortunate that you've had to rebase a number of times, but the other conflicting changes were higher priority, and we don't want to merge random refactors in between bug fixes, that need to be backported, as that only pointlessly complicates the backporting 14:50:31 <achow101> #endmeeting