16:00:22 <achow101> #startmeeting 
16:00:22 <corebot> achow101: Meeting started at 2025-07-31T16:00+0000
16:00:23 <corebot> achow101: Current chairs: achow101
16:00:24 <corebot> achow101: Useful commands: #action #info #idea #link #topic #motion #vote #close #endmeeting
16:00:25 <corebot> achow101: See also: https://hcoop-meetbot.readthedocs.io/en/stable/
16:00:26 <corebot> achow101: Participants should now identify themselves with '#here' or with an alias like '#here FirstLast'
16:00:29 <rkrux> hi
16:00:29 <TheCharlatan> hi
16:00:30 <janb84> hi
16:00:32 <achow101> #bitcoin -core-dev Meeting: abubakarsadiq achow101 _aj_ ajonas b10c brunoerg cfields darosior dergoegge fanquake fjahr furszy gleb glozow hebasto hodlinator instagibbs jarolrod jonatack josibake kanzure laanwj LarryRuane lightlike luke-jr maflcko marcofleon maxedw Murch pinheadmz provoostenator ryanofsky sdaftuar S3RK stickies-v sipa sr_gi tdb3 theStack TheCharlatan vasild willcl-ark
16:00:33 <pinheadmz> hi
16:00:35 <cfields> hi
16:00:37 <Sjors[m]1> hi
16:00:38 <lightlike> hi
16:00:39 <kevkevin> hi
16:00:44 <furszy> hi
16:00:55 <achow101> There are 2 preproposed meeting topics this week. Any last minute ones to add?
16:01:12 <emzy> hi
16:01:17 <eugenesiegel> hi
16:01:21 <l0rinc> hi
16:01:21 <johnny9dev> hi
16:01:30 <willcl-ark> hi
16:01:30 <Murch[m]> hello
16:01:59 <achow101> #topic Kernel WG Update (TheCharlatan)
16:02:25 <TheCharlatan> There's some movement happening again on the API PR, currently scoping out approaches for making usage safer with reference counting.
16:02:35 <darosior> hi
16:03:16 <stickies-v> hi
16:03:44 <TheCharlatan> it's also been attracting some new people to contribute, which is nice :)
16:04:08 <b10c> hi
16:04:17 <TheCharlatan> might having something more concrete to share again in a few weeks.
16:04:20 <TheCharlatan> that's all
16:04:46 <achow101> #topic Stratum v2 WG Update (sjors)
16:05:55 <glozow> hi
16:06:48 <achow101> from sjors: Same as last week, would still like to see #31679 and #31802 make it for the feature freeze.
16:06:51 <corebot> https://github.com/bitcoin/bitcoin/issues/31679 | cmake: Move internal binaries from bin/ to libexec/ by ryanofsky · Pull Request #31679 · bitcoin/bitcoin · GitHub
16:06:53 <corebot> https://github.com/bitcoin/bitcoin/issues/31802 | Add bitcoin-{node,gui} to release binaries for IPC by Sjors · Pull Request #31802 · bitcoin/bitcoin · GitHub
16:07:01 <achow101> #topic MuSig2 WG Update (achow101)
16:07:40 <achow101> No changes since last week, #31244 is still probably rfm. I've pulled in several of the suggested followups into #29675
16:07:44 <corebot> https://github.com/bitcoin/bitcoin/issues/31244 | descriptors: MuSig2 by achow101 · Pull Request #31244 · bitcoin/bitcoin · GitHub
16:07:45 <corebot> https://github.com/bitcoin/bitcoin/issues/29675 | wallet: Be able to receive and spend inputs involving MuSig2 aggregate keys by achow101 · Pull Request #29675 · bitcoin/bitcoin · GitHub
16:07:59 <achow101> #topic orphan resolution WG Update (glozow)
16:08:25 <glozow> still looking to get #32941 in for v30
16:08:27 <corebot> https://github.com/bitcoin/bitcoin/issues/32941 | p2p: TxOrphanage revamp cleanups by glozow · Pull Request #32941 · bitcoin/bitcoin · GitHub
16:08:46 <glozow> that's all
16:08:57 <achow101> #topic QML GUI WG Update (jarolrod, johnny9dev)
16:09:09 <kanzure> hi
16:09:41 <johnny9dev> We're making good progress on patching remaining issues with moving the project to submodule, cmake, and qt6. Deer gee has ported over his AssumeUTXO PR over to the new structure already (bitcoin-core/gui-qml#485) . I should have my previous PRs ported over this weekend.
16:09:43 <corebot> https://github.com/bitcoin-core/gui-qml/issues/485 | QML Load UTXO Snapshot by D33r-Gee · Pull Request #485 · bitcoin-core/gui-qml · GitHub
16:09:57 <phantomcircuit> i've been looking at initial block download times and noticed that we're recalculating block header hashes many many times (like 800+ times in cases), this can be fixed forever with an immutable block/header class(es)
16:10:06 <phantomcircuit> does anybody have thoughts on such an endeavor
16:10:21 <pinheadmz> i believe that PR will also require an upstream PR to expose some utxo snapshot thing in the interface
16:10:31 <fjahr> hi
16:10:53 <johnny9dev> It will. I suggested we maintain a patch until then
16:10:54 <pinheadmz> so thatll be the new, somewhat frustrating, process but i think the separation of interests there is good
16:11:07 <achow101> phantomcircuit: we're in a meeting, can add that as a topic if you want
16:11:08 <l0rinc> phantomcircuit: I have investigated that before, but don't have exact numbers
16:11:15 <darosior> phantomcircuit: interesting, but let's wait till other prepared meeting topics are done?
16:11:21 <Murch[m]> achow101: How long as #32144 been RFM when you say "it’s still probably rfm"?
16:11:24 <corebot> https://github.com/bitcoin/bitcoin/issues/32144 | lint: Remove needless borrow to fix Clippy warning by strmfos · Pull Request #32144 · bitcoin/bitcoin · GitHub
16:11:33 <phantomcircuit> achow101, yeah i don't know how meeting works anymore, that was a request for it to be a topic
16:12:00 <johnny9dev> Anyway we're almost caught up with previous functionality. That's all from gui-qml
16:12:12 <achow101> Murch[m]: 3 or 4 weeks
16:12:42 <darosior> johnny9dev: nice
16:12:43 <Murch[m]> achow101: Any idea who should add their review or take a look at that for it to get merged?
16:14:09 <Murch[m]> Sorry, #31244, not what I wrote
16:14:12 <corebot> https://github.com/bitcoin/bitcoin/issues/31244 | descriptors: MuSig2 by achow101 · Pull Request #31244 · bitcoin/bitcoin · GitHub
16:14:19 <achow101> Murch[m]: anyone who is familiar with descriptors, or it could be merged
16:14:54 <Murch[m]> It does have ACKs from w0xlt, rkrux, theStack, and Sjors
16:15:35 <achow101> #topic dbwrapper read/write asymmetry (l0rinc)
16:15:44 <l0rinc> dbwrapper writes currently pretend to return bool but always return true, while real errors surface via exceptions – #33042 makes that explicit, but we need to decide the canonical error path
16:15:45 <corebot> https://github.com/bitcoin/bitcoin/issues/33042 | refactor: inline constant return values from `dbwrapper` write methods by l0rinc · Pull Request #33042 · bitcoin/bitcoin · GitHub
16:16:42 <l0rinc> `read` catches the exception and returns, but `write` always returned `true`, while throwing in the background
16:17:04 <achow101> how do errors currently propagate? crashing everything?
16:17:12 <l0rinc> most of the time yes
16:17:33 <TheCharlatan> they are eventually converted to FatalError's yeah
16:18:11 <l0rinc> is the removal of the fake return value enough here or do we need to unify the read/write interfaces?
16:18:35 <achow101> if we can't write state to disk, that does seem like a situation where we'd want to abort anyways
16:18:53 <glozow> Murch: achow101: will have a look at #31244 later today
16:18:56 <corebot> https://github.com/bitcoin/bitcoin/issues/31244 | descriptors: MuSig2 by achow101 · Pull Request #31244 · bitcoin/bitcoin · GitHub
16:19:17 <achow101> if the return value is never used, then we should not return it
16:19:35 <TheCharlatan> rewriting it to propagate an error code seems riskier than just removing what is essentially dead code.
16:19:58 <rkrux> glozow: thanks
16:20:16 <l0rinc> thanks for the feedback, I'll add these comments to the PR - unless the authors want to
16:20:51 <darosior> TheCharlatan: +1
16:21:03 <achow101> #topic CI migration to Cirrus Runners (willcl-ark)
16:21:39 <willcl-ark> Myself and @m3dwards have been working on a change to migrate to our CI to hosted Cirrus Runners. This is now open in #32989.
16:21:42 <corebot> https://github.com/bitcoin/bitcoin/issues/32989 | ci: Migrate CI to hosted Cirrus Runners by willcl-ark · Pull Request #32989 · bitcoin/bitcoin · GitHub
16:21:46 <willcl-ark> The benefits and various tradeoffs of such a move are detailed in the top comment in the PR, but in summary it should make the CI maintainable by more people, not be reliant on the current infrastructure, and also provide "easier scaling"; if you want to scale more machines, just throw more $$ at the problem. These are fleshed out further in the PR, and we'd welcome feedback there.
16:21:57 <achow101> ack
16:21:59 <willcl-ark> Moving to our own runners also has a knock-on effect that all the (current) "cirrus" jobs will run on your own forks slower, on GH free runners, so self-hosting a cirrus runner may no longer be necessary if speed it not an issue for you. Forks (like inquisition) would be able to buy their own runners if they wanted faster runners for their repos too, and enable with a trivial patch.
16:22:11 <willcl-ark> I wanted to highlight it in a meeting just to try and alert as many people to it as possible, in case there was any opposition to it which hadn't been raised yet. Naturally, we are also looking for more reviewers too! So if anyone has any interest, then feel free to consider this a review-beg as well :)
16:22:41 <willcl-ark> That's all from me on that topic, unless anyone has any questions they'd like to ask about it here.
16:23:20 <achow101> #topic ibd recalculating block hashes (phantomcircuit)
16:24:24 <achow101> we don't store the hash in CBlockHeader?
16:24:35 <phantomcircuit> so i instrumented cblockheader::gethash to count how many times we reclculate the headers hash and i got some extreme outliers like 800+ times, i think this has to do with receiving blocks out of order during IBD and the behavior of activatebestchain
16:26:15 <achow101> If there's a measuable performance improvement in IBD, then I think that seems like a good idea
16:26:18 <phantomcircuit> achow101, so the block headers are about the same length as the hash and we reference the block headers with the hash, so in most places we should already have the hash and know that we haven't changed it. though the logic for that doesn't flow through everywhere, so we end up calling gethash more than we should
16:26:18 <l0rinc> phantomcircuit: I can take another look at it, though making blocks mostly immutable isn't trivial
16:27:28 <darosior> sha256 over 80 bytes of data should be quite fast. But yeah if we do indeed do it 800+ times (!!) for every single header in the chain that does sound like it could speed things up.
16:27:41 <phantomcircuit> it trades memory usage for performance but the dbcache is already huge so whatever, an additional 45mb of memory usage to cache the block header hashes along with the the headers
16:28:06 <achow101> hmm, that is 10% of default dbcache though
16:28:17 <darosior> That sounds non-trivial
16:28:24 <phantomcircuit> we usually do it several times per header but in the perverse case it is 800+ times
16:28:49 <l0rinc> I've pushed a few changes deduplicating hash calculations already - should be slightly less than that now. Are you saying that the exact same block has 800+ recalculations?
16:28:51 <fjahr> how often have you seen the perverse case?
16:28:54 <darosior> ceteris paribus i expect 45mb of dbcache to speed IBD more so than caching a sha256 over 80 bytes
16:28:55 <phantomcircuit> it depends on the behavior of activatebestchain when the blocks are received out of order, I think
16:29:19 <achow101> if we already know the hash elsewhere, can we use it in more places rather than caching the hash again?
16:29:20 <phantomcircuit> l0rinc, yes, one block header has 800 recalculations
16:29:50 <phantomcircuit> fjahr, my graphs suck so I don't know, but it was bad
16:30:02 <darosior> I'd be very curious to learn more about the situation in which we re-calculate the same header hash 800 times
16:30:07 <l0rinc> We can likely cache some of those locally, I tried making the blocks immutable and it was quite messy
16:30:15 <TheCharlatan> achow101 that seems like my intuition too. After all we do save it in the index with every header already.
16:30:58 <phantomcircuit> darosior, it's dependent on the out of order activatebestchain, so it won't show in normal unit tests or anything
16:31:17 <l0rinc> I have doubts about the 800 recalculations, seems like a miscalculation to me (I have also investigated this and haven't found anything like that) - but we can do that after the meeting as well
16:31:37 <lightlike> why 45MB? isn't the CBlockHeader objects freed once the block is no longer in memory (other than the CBlockIndex)?
16:32:17 <phantomcircuit> l0rinc, it's annoying and you won't see the behavior with blocks received in order
16:32:24 <achow101> it definitely seems there's something to look at here. phantomcircuit I suggest you work with l0rinc and the others who have also been looking at ibd improvements and benchmarking
16:32:56 <darosior> +1
16:33:29 <achow101> #topic Benchmarking WG Update (josie, l0rinc)
16:33:33 <furszy> memory shouldn't be a concern, could directly use the phashBlock field from the index.
16:33:35 <phantomcircuit> the way to have the immutable header and block classes is to have it silently convert between the two with constructors
16:33:43 <achow101> (missed this during the wg update section)
16:33:47 <l0rinc> I have doubts about the 800 recalculations, seems like a miscalculation to me (I have also investigated this and haven't found anything like that) - but we can do that after the meeting as well
16:33:53 <phantomcircuit> then slowly change from mutable to immutable everywhere
16:33:53 <l0rinc> sorry, wrong message
16:34:01 <l0rinc> #31144 and #32279 were just merged, we're making good progress. The next important ones where reviewers are needed are:
16:34:05 <corebot> https://github.com/bitcoin/bitcoin/issues/31144 | [IBD] multi-byte block obfuscation by l0rinc · Pull Request #31144 · bitcoin/bitcoin · GitHub
16:34:07 <corebot> https://github.com/bitcoin/bitcoin/issues/32279 | [IBD] prevector: store `P2WSH`/`P2TR`/`P2PK` scripts inline by l0rinc · Pull Request #32279 · bitcoin/bitcoin · GitHub
16:34:16 <l0rinc> #31645 The batch size for UTXO set writes is now calculated based on the maximum dbcache size to ensure that with the default values, memory usage doesn't increase, while reducing flushing time when there is enough memory available.
16:34:16 <l0rinc> The change reduces the IBD time by a fixed amount, it's speeding up a critical part of saving the state for long-term storage.
16:34:18 <corebot> https://github.com/bitcoin/bitcoin/issues/31645 | [IBD] flush UTXO set in batches proportional to `dbcache` size by l0rinc · Pull Request #31645 · bitcoin/bitcoin · GitHub
16:34:29 <l0rinc> #32497 Set accurate capacity for Merkle root calculation to avoid reallocations
16:34:31 <corebot> https://github.com/bitcoin/bitcoin/issues/32497 | merkle: pre‑reserve leaves to prevent reallocs with odd vtx count by l0rinc · Pull Request #32497 · bitcoin/bitcoin · GitHub
16:35:13 <achow101> l0rinc: can you suggest a specific one people should focus on first?
16:35:27 <l0rinc> one is very simple  #32497
16:35:29 <corebot> https://github.com/bitcoin/bitcoin/issues/32497 | merkle: pre‑reserve leaves to prevent reallocs with odd vtx count by l0rinc · Pull Request #32497 · bitcoin/bitcoin · GitHub
16:35:50 <l0rinc> the other one needs some reproducers #31645 - any help would be appreciated in both
16:35:54 <corebot> https://github.com/bitcoin/bitcoin/issues/31645 | [IBD] flush UTXO set in batches proportional to `dbcache` size by l0rinc · Pull Request #31645 · bitcoin/bitcoin · GitHub
16:37:02 <achow101> Any other topics to discuss this week?
16:38:16 <achow101> #endmeeting