14:00:21 <achow101> #startmeeting 14:00:23 <sdaftuar> hi 14:00:25 <stickies-v> hi 14:00:27 <fjahr> hi 14:00:29 <pinheadmz> yo 14:00:30 <cfields> hi 14:00:32 <hebasto> hi 14:00:35 <achow101> #bitcoin -core-dev Meeting: achow101 _aj_ amiti ariard aureleoules b10c BlueMatt brunoerg cfields darosior dergoegge dongcarl fanquake fjahr furszy gleb glozow hebasto instagibbs jamesob jarolrod jonatack josibake kallewoof kanzure kouloumos kvaciral laanwj LarryRuane lightlike luke-jr MacroFake Murch phantomcircuit pinheadmz promag provoostenator ryanofsky sdaftuar S3RK stickies-v sipa theStack TheCharlatan vasild 14:00:37 <glozow> hi 14:00:41 <brunoerg> hi 14:00:41 <vasild> hi 14:00:43 <furszy> hi 14:00:50 <achow101> There is one pre-proposed meeting topic this week. Any last minute ones to add? 14:00:52 <b10c> hi 14:00:55 <Murch[m]> hi 14:00:55 <dergoegge> hi 14:01:13 <virtu> hi 14:01:34 <achow101> #topic package relay updates (glozow) 14:01:50 <glozow> #28970 is the priority. (thanks everyone for the reviews, will push later today) 14:01:54 <gribble> https://github.com/bitcoin/bitcoin/issues/28970 | p2p: opportunistically accept 1-parent-1-child packages by glozow ÷ Pull Request #28970 ÷ bitcoin/bitcoin ÷ GitHub 14:01:58 <tdb3> hi 14:02:11 <laanwj> hi 14:02:40 <josie> hi 14:02:55 <glozow> And I'm sure instagibbs would be happy for reviews on #28984 14:02:57 <gribble> https://github.com/bitcoin/bitcoin/issues/28984 | Cluster size 2 package rbf by instagibbs ÷ Pull Request #28984 ÷ bitcoin/bitcoin ÷ GitHub 14:03:08 <glozow> That's all from me 14:03:29 <achow101> #topic cluster mempool updates (sdaftuar) 14:03:31 <darosior> hi 14:03:43 <sipa> hi 14:04:01 <sdaftuar> I'm almost done rewriting my branch with the draft cluster mempool implementation, which I'll then rebase on master and push up to the draft PR. 14:04:37 <sdaftuar> I believe Pieter is working on getting a version of the linearization code ready for its own PR. Once we're at that point, that would be the first item for people to review as part of the cluster mempool roadmap. 14:04:43 <willcl-ark> Hi 14:04:50 <kanzure> hi 14:05:42 <sdaftuar> Also, I posted a summary of my research on data from 2023 to delving, summarizing the effects of cluster mempool on last year's data. If anyone has further thoughts/concerns about the effects, I'd love to hear feedback. 14:05:51 <sdaftuar> I think that's it for me 14:05:53 <achow101> is there an eta on that PR being opened? 14:06:27 <sdaftuar> not sure, sipa? 14:06:44 <sipa> i hope in the next week or so 14:07:43 <achow101> looking forward to it 14:08:00 <achow101> #topic legacy wallet removal updates (achow101) 14:08:38 <bitcoin-git> [bitcoin] BorjaPractica opened pull request #29905: modificación (24.x...master) https://github.com/bitcoin/bitcoin/pull/29905 14:08:45 <achow101> #26606 is still the pr to review, and it's been getting some review since the session we had at CoreDev. I'm hoping that it can be merged in the next couple of weeks 14:08:47 <gribble> https://github.com/bitcoin/bitcoin/issues/26606 | wallet: Implement independent BDB parser by achow101 ÷ Pull Request #26606 ÷ bitcoin/bitcoin ÷ GitHub 14:09:13 <achow101> otherwise, #28574 is also a good target for review 14:09:15 <gribble> https://github.com/bitcoin/bitcoin/issues/28574 | wallet: optimize migration process, batch db transactions by furszy ÷ Pull Request #28574 ÷ bitcoin/bitcoin ÷ GitHub 14:09:35 <bitcoin-git> [bitcoin] fanquake closed pull request #29905: modificación (24.x...master) https://github.com/bitcoin/bitcoin/pull/29905 14:10:22 <achow101> if people have weird legacy wallets, testing out #26606 on them would be great 14:10:25 <gribble> https://github.com/bitcoin/bitcoin/issues/26606 | wallet: Implement independent BDB parser by achow101 ÷ Pull Request #26606 ÷ bitcoin/bitcoin ÷ GitHub 14:10:46 <achow101> #topic Ad-hoc high priority for review 14:10:50 <achow101> Anything to add or remove from https://github.com/orgs/bitcoin/projects/1/views/4 14:11:39 <sipa> i'd like to get some eyes on #29625 14:11:41 <gribble> https://github.com/bitcoin/bitcoin/issues/29625 | Several randomness improvements by sipa ÷ Pull Request #29625 ÷ bitcoin/bitcoin ÷ GitHub 14:12:06 <laanwj> still planning to do some big endian tests with the migration wallet 14:12:23 <achow101> sipa: added 14:12:32 <sipa> it gives us an actual fully deterministic mode for the RNG for use in fuzz and tests, hopefully dropping the need for various "deterministic mode" things in the future 14:13:16 <sipa> achow101: thanks 14:13:28 <achow101> #topic libevent: replace? fork & maintain? leave it alone? 14:13:43 <achow101> (pinheadmz) 14:13:46 <pinheadmz> yeah so i started working on adding unix socket support for rpc 14:14:02 <pinheadmz> libevent claims it supports unix sockets, but it dont: https://github.com/libevent/libevent/issues/1615 14:14:32 <pinheadmz> we pretty much only use libevent for http service, in the rpc server and bitcoin cli 14:14:48 <cfields> concept ack :) 14:14:54 <pinheadmz> and if you look at libevent repo, its all bitcoin core devs issues and prs anyway 14:14:57 <achow101> my super naive thought is that since we already do the socket stuff for p2p handling, and http is a fairly simple and straightforward protocol (especially for our usage), it wouldn't be that complicated to implement it ourselves? 14:15:05 <fjahr> Strongly in favor of removing libevent and replacing it with our own code. My frustrations with it are a few years old already (https://github.com/libevent/libevent/issues/1024) and there are many others that have felt the same before me and since. My mind went there very quickly also after reading the zx story, pretty much exact same symptoms can be observed in libevent. I donâÂÂt think we want to maintain a fork. I think 14:15:05 <fjahr> implementing the libevent parts we actually use are not an easy project but sets us up much better for the future. 14:15:35 <achow101> I could be totally off base though if anyone actually has experience with writing http servers 14:15:37 <laanwj> concept ack on replacing libevent with something of our own, not with moving to another event library like libuv, would be a large chance of the same shit again 14:15:39 <pinheadmz> does anyone know off hand a compact simple http server written in cpp with MIT license? 14:16:05 <cfields> many many years ago I tried to port our net code to libevent rather than handling sockets ourselves and found it to be woefully underpowered and unmaintained back then. 14:16:23 <cfields> imo it's a substantial liability at this point 14:16:30 <pinheadmz> ok so sounds like favor is to inlude new code in the repo. easiest would be to find an existing implementation with compatible license and vendor it i suppose 14:16:33 <stickies-v> at the previous coredev cfields and I had a look at potential options to replace the libevent webserver and https://github.com/uNetworking/uWebSockets seemed like a potential candidate, its C++17 and seems quite well maintained 14:16:39 <stickies-v> it's apache 2.0 license tho 14:16:42 <laanwj> a http server is doable if you can keep the scope small, internal-use only, no high perf requirements, it can't be nginx or apache 14:17:20 <pinheadmz> right, and if we deprecate the REST interface i think we dont even need to support GET requests ;-) 14:17:23 <vasild> Concept ACK on replacing the sockets stuff with our own. Maybe the http server as well. 14:17:48 <pinheadmz> vasild p2p sockets are already our own right? is that what you mean 14:17:59 <sipa> stickies-v: websockets sounds like about an order of magnitude harder than just http, actually... 14:18:44 <pinheadmz> who did the work to replace boost::process ? wondering how that kind of thing goes, replacing a big dependency 14:18:47 <achow101> sipa: I had a look at websockets a few months ago (for "serverless" payjoin I think) and it's actually not that complicated 14:19:07 <pinheadmz> achow101 websockets still requires http... 14:19:07 <laanwj> yes, websockets are more involved, but not much 14:19:19 <darosior> Except for backward compat, why does JSONRPC has to be through HTTP at all? 14:19:20 <fjahr> has anyone had experience with https://github.com/yhirose/cpp-httplib? 14:19:22 <laanwj> but we're not talking about changing the RPC protocol here 14:19:26 <sipa> okay, i know too little about webstuff to really have an informed opinion 14:19:29 <sipa> laanwj: yeah 14:19:30 <laanwj> let's keep that orthogonal 14:19:37 <hebasto> pinheadmz: work of still in progress; unused code has to be removed 14:19:38 <vasild> pinheadmz: I mean to replace all libevent-sockets with our own code - talking to the socks5 proxy. Replacing the libevent http server would be a separate bigger task, maybe feasible as well. 14:20:06 <darosior> (c-lightning for instance has its JSONRPC server directly without HTTP) 14:20:16 <achow101> darosior: because it's always been that way :p 14:20:24 <darosior> :) 14:20:48 <sipa> maybe together with a move to UNIX-socket based RPC, we can drop the HTTP part? 14:20:53 <pinheadmz> darosior super interesting idea 14:21:02 <pinheadmz> so users cant for example curl the api 14:21:07 <sipa> (we'd keep the HTTP part for TCP-based JSON-RPC, of course) 14:21:38 <achow101> sipa: I'd rather not. there's a ton of stuff that already uses HTTP, and switching from tcp to unix sockets as a client isn't particularly complicated 14:21:56 <fanquake> what is the ton of stuff 14:22:04 <sipa> achow101: right, but that stuff has to change anyway if it wants to use UNIX sockets 14:22:08 <cfields> could be a proxy http app in between? 14:22:15 <laanwj> agree with achow101, so much stuff is using http 14:22:30 <achow101> fanquake: literally anything that's already an rpc client? e.g. lightning impls, various python, rust, other language libraries 14:22:31 <pinheadmz> i think we have to support http forever right? its not just bitcoin-cli its the client lib written in every language 14:22:33 <pinheadmz> ueaj 14:22:36 <pinheadmz> s/yeah 14:22:49 <laanwj> in retrospect it'd have been better to do like c-lightning, just JSON records over a pipe, but i don't think it's a good idea to change that now 14:22:50 <sipa> it was just an idea, not a strong opinion 14:23:04 <laanwj> all user software uses the current JSON RPC, if we abandon that, no one will upgrade. anymore basically :) 14:23:31 <pinheadmz> ok so sounds like the move is to include a http.cpp in bitcoin core one way or another 14:23:49 <sipa> my idea is that we'd have two interfaces, TCP+HTTP+JSON-RPC, and UNIX+JSON-RPC, and both remain supported forever 14:23:55 <pinheadmz> doesnt need to be the best webserver ever but should handle long ques of rpc commands from multiple clients 14:24:03 <laanwj> heck there's software that still requires the legacy wallet, i intend to help joinmarket port to the new one, interface drift is a pain even where it's really needed 14:24:36 <pinheadmz> sipa well there is also a broader interest in not getting xz-ed by libevent 14:24:49 <laanwj> @pinheadmz exactly 14:24:54 <_aj_> pinheadmz: (can't get xz'ed if upstream never updates though?) 14:25:03 <pinheadmz> lol there is that 14:25:05 <fanquake> I think it's much less likely to happen via libevent than other deps 14:25:26 <fanquake> and it basically already happened via boost process 14:25:45 <pinheadmz> dont wanna stray but did boost get hacked ? 14:25:55 <fanquake> compromising one of N random boost maintainers to get something into the 130mb tarball is much easier than getting something into libevent, that doesn't do new releases 14:26:06 <achow101> pinheadmz: I think the next step would be to concretely propose a replacement then, as there seems to be agreement that replacing with something is the way to go. 14:26:09 <laanwj> i was kind of disappointed in the maintenance state of libevent, back when i tried to add UNIX socket support 14:26:22 <pinheadmz> laanwj i got chu fam 14:26:25 <darosior> achow101: +1 14:26:31 <pinheadmz> achow101 agreed 14:26:38 <laanwj> @achow101 agree 14:26:40 <fanquake> pinheadz: not hacked, when we re-enabled boost-process for windows it was doing random networking things, that never got picked up in review 14:27:03 <pinheadmz> ð¬ 14:27:09 <laanwj> windows has this wacky winsock library that should only be initialized by us 14:27:20 <achow101> anything else to discuss today? 14:27:20 <hebasto> yeap 14:27:38 <fjahr> If you read the timeline of the xz backdoor, it exactly sounds like the state of libevent now, if libevent started to do releases more often all of a sudden I would be very scared 14:28:03 <laanwj> @fjahr libevent is an attractive target because of its use in tor 14:28:23 <fanquake> Yea it's used all over the place. chromium, torrent libs etc 14:28:43 <laanwj> that also makes that there are probably lots of security conscious people looking at it though 14:29:01 <vasild> "if libevent started to do releases more often all of a sudden" -- or it has already been backdoored and things have settled now... 14:29:08 <laanwj> heeh 14:29:24 <achow101> #endmeeting