19:00:22 <achow101> #startmeeting 19:00:23 <core-meetingbot> Meeting started Thu Feb 2 19:00:22 2023 UTC. The chair is achow101. Information about MeetBot at https://bitcoin.jonasschnelli.ch/ircmeetings. 19:00:23 <core-meetingbot> Available commands: action commands idea info link nick 19:00:23 <brunoerg> hi 19:00:26 <achow101> #bitcoin -core-dev Meeting: achow101 aj amiti ariard b10c BlueMatt cfields Chris_Stewart_5 darosior digi_james dongcarl elichai2 emilengler fanquake fjahr gleb glozow gmaxwell gwillen hebasto instagibbs jamesob jarolrod jb55 jeremyrubin jl2012 jnewbery jonasschnelli jonatack jtimon kallewoof kanzure kvaciral laanwj larryruane lightlike luke-jr maaku marcofalke meshcollider michagogo moneyball morcos nehan NicolasDorier paveljanik petertodd 19:00:26 <achow101> phantomcircuit promag provoostenator ryanofsky sdaftuar sipa vasild 19:00:29 <hebasto> hi 19:00:32 <fjahr> hi 19:00:33 <jon_atack> hi 19:00:36 <furszy> hi 19:00:51 <lightlike> hi 19:01:06 <achow101> There are two preproposed meeting topics: ASMap in the release process (fjahr), Self-hosted Gitlab instance (fjahr) 19:01:13 <achow101> anything else to add to the list? 19:01:25 <glozow> hi 19:01:28 <pinheadmz> hi 19:01:43 <achow101> #topic high priority for review 19:01:43 <core-meetingbot> topic: high priority for review 19:01:48 <MacroFake> for me plz: https://github.com/bitcoin-core/gui/pull/697 (Remove reindex special case from the progress bar label) 19:02:02 <achow101> https://github.com/orgs/bitcoin/projects/1 19:02:02 <LarryRuane> hi 19:02:07 <achow101> anything to add/remove/merge? 19:02:28 <kanzure> hi 19:03:00 <achow101> MacroFake: done 19:03:04 <MacroFake> thanks! 19:04:11 <achow101> #22693 and #25740 are both recently needs rebase 19:04:15 <gribble> https://github.com/bitcoin/bitcoin/issues/22693 | RPC/Wallet: Add "use_txids" to output of getaddressinfo by luke-jr ÷ Pull Request #22693 ÷ bitcoin/bitcoin ÷ GitHub 19:04:18 <gribble> https://github.com/bitcoin/bitcoin/issues/25740 | assumeutxo: background validation completion by jamesob ÷ Pull Request #25740 ÷ bitcoin/bitcoin ÷ GitHub 19:05:56 <achow101> #topic ASMap in the release process (fjahr) 19:05:57 <core-meetingbot> topic: ASMap in the release process (fjahr) 19:06:03 <fjahr> Hi, first of all, sorry about the wordy write-up, I should have included a tldr; I just wrote a quick one for those that didnâÂÂt have the time to read it. The main points I am making are: 1. In order to ship an asmap file with the release we want to make sure itâÂÂs the best quality data. The main way to QA was assumed to be based on diffing files. I tried to find a systematical way to do QA via diffs so I donâÂÂt think it 19:06:03 <fjahr> is a reliable option. 2. Instead we should make the asmap creation process transparent and reproducible. The asmap file should be treated like a PR, it will be presented and reviewers can test and run QA on it as they like but that is not part of the release process or responsibility of maintainer. 3. Quality of the input data is even more important. I believe that for our purpose rpki > irr > collectors, in terms of 19:06:03 <fjahr> quality. So far only collectors were considered as an input source. 19:06:27 <fjahr> Thanks to everyone who provided feedback so far! There was some nice feedback from sipa here earlier today, it shows that in the process of joining the input data the devil is in the details and there is still more research to be done but I think itâÂÂs moving in the right direction. Also my code is buggy ^^ 19:06:51 <achow101> doesn't getting asmap data require having your own AS? 19:07:00 <fjahr> I would like to answer any questions and take feedback on what is bothering people. 19:07:40 <sipa> hi, i'm half here 19:07:50 <fjahr> achow101: there are lots of open sources for data. the approaches we are discussing are all built on those. 19:08:38 <achow101> i should probably read the backscroll 19:08:42 <fjahr> But if you have any way to open a BGP session, you can collect the global feed youself and for some people that may be a preferred aproach 19:09:00 <fjahr> But most people probably won't be able to do it 19:09:23 <fjahr> Achow101: the "general knowledge" section in the write-up should be interesting :) 19:09:43 <fjahr> https://gist.github.com/fjahr/bf0ff0917e03a4e49fac0617b2b35747 19:09:51 <sipa> As I understand it, there are 2 main ways of building the data: (a) from BGP routes (which requires being your own BGP node, or getting trusted data from someone dumping it) or (b) authoritatively from the assigmed mappings directly. 19:09:52 <brunoerg> > The main way to QA was assumed to be based on diffing files 19:10:05 <achow101> #link https://gist.github.com/fjahr/bf0ff0917e03a4e49fac0617b2b35747 19:10:20 <brunoerg> I agree, but what should we compare? 19:11:14 <fjahr> My opinion the diffing question: ThatâÂÂs probably the most controversial point among those that have dove deeper into the topic. In short: I just couldnâÂÂt find any way to develop a concise set of rules that could be giving a clear indication if a file is safe/unsafe and I tried really hard (TM). So I think if we tell people they can run a diff and there is a X% match on metric Y that means itâÂÂs safe, then we give 19:11:14 <fjahr> people a false sense of security at best, at worst the release process becomes a mess because people are unsure if the numbers mean the file is safe or not and then nobody ends up using it. I am happy to review if people have suggestions for this but if we wait for this then the process will drag on for a while IMO. 19:11:41 <sipa> FWIW, I concur that diffing isn't viable as a means of QA (or not the primary one at least), the differences from maps constructed from one day to another often have substantial changes already (more than you as a human could look at and judge "this looks reasonable" or not). 19:11:43 <fjahr> So, I think "we" as in as a part of the release process shouldn't compare anything IMO 19:12:15 <lightlike> fjahr: but what should a reviewer do in order to ack the PR? 19:12:20 <fjahr> But anyone can run experiments, that's what the review phase is for 19:12:38 <brunoerg> I made an experiment generating 2 files from 2 last Core's release date and there was A LOT of changes 19:12:48 <fjahr> lightlike: that's up to the reviewer, I mean I don't tell people how to review my PRs 19:13:02 <fjahr> of course there will be knowledge sharing and tools written for this 19:13:20 <fjahr> but people can not expect Bitcoin core maintainers to do that work for them 19:13:49 <sipa> As I understand it, @fjahr's suggesting is that we effectively establish a reproducible process for building the maps, so hopefully it suffices for some reviewers to actually reproduce it 19:13:55 <fjahr> brunoerg: yepp, and that's the problem 19:14:05 <sipa> Though what that process would be seems a bit unspecified for now? 19:14:26 <brunoerg> @fjahr: but in that PR you would tell how you generated that file in order to reviewers to follow the same step? 19:15:16 <fjahr> sipa: will need to be specified a lot more in detail, yes 19:15:35 <fjahr> brunoerg: from the same raw files, yes 19:15:36 <jamesob> achow101: thanks for the ping, I'll rebase AU momentarily 19:15:48 <brunoerg> cool 19:16:01 <fjahr> the raw files are also uploaded in the zip file 19:17:01 <achow101> that doesn't really solve the problem though, the raw files could just be malicious 19:17:42 <sipa> Maybe a bit more practical, do you expect that the txt range/asn map file will become part of the bitcoin core source repo? or just the combined binary dat file? or neither? 19:18:05 <sipa> binary file is roughly 1.2 MB right now 19:18:07 <fjahr> For the RPKI data the only way it could be malicious is there could be stuff missing 19:18:23 <fjahr> everything else can be malicious, yes, but that the general crux with BGP 19:19:05 <fjahr> so you can be Amazon or the NSA and still end up with a BGP hijack in your routing table 19:19:49 <sipa> RPKI is signed, right? 19:20:17 <fjahr> sipa: I think not in the source, I mean it's possible, but it seems more elegant to have the candidates in asmap-data seperately and only join it during build process 19:20:23 <fjahr> sipa: yes 19:21:01 <achow101> do we expect asmap to be in our release distributions? 19:21:13 <fjahr> The easiest way to explain it is: RPKI is SSL for BGP (very roughly) 19:21:39 <fjahr> achow101: that is kind of the goal 19:22:06 <sipa> @achow101 My hope is that eventually it'll end up being part of the binary, or a separately-distributed file that's part of the installation and used by default 19:22:07 <fjahr> *end goal, there may be intermittend steps where it's not the default etc. 19:23:21 <fjahr> The one downside of RPKI is it's not deployed everywhere so it can not be the only data source 19:24:23 <sipa> but we could use RPKI as primary, and only use other sources for ranges for which RPKI is not available? 19:24:37 <fjahr> yes, that is the approach I build with kartograf 19:25:22 <fjahr> It's a POC: https://github.com/fjahr/kartograf 19:27:10 <fjahr> A few people have asked this: yes, I will also post this to the ML soon, just wanted to hear from this group first, particular potential pushback on the release process stuff 19:28:53 <fjahr> Any more questions? I am happy to answer at any time here or in DMs, I hope we can get as many people onboard on this problem space as possible to get more review :) 19:30:05 <achow101> thanks for reviving this project. I'll have to do some more reading before I can give an feedback 19:30:28 <achow101> #topic Self-hosted Gitlab instance (fjahr) 19:30:29 <fjahr> achow101: great, thanks! 19:30:29 <core-meetingbot> topic: Self-hosted Gitlab instance (fjahr) 19:30:46 <fjahr> Err, basically I just wanted to repeat what I said earlier: Just a quick announcement/invite if someone wants to get access and try stuff out, also happy to share admin access with the people I know, itâÂÂs a playground. Currently some key features are missing because itâÂÂs on the Community Plan but there are open source specific licenses and Mike Schmidt has been in contact with them already about an Enterprise license, 19:30:47 <fjahr> weâÂÂll probably coordinate more on that soon. But I am a more focussed on ASMap for now tbh. 19:31:03 <fjahr> ItâÂÂs here: gitlab.sighash.org, but requires registration to see anything currently. No need for an email confirmation so you can enter whatever you want. Just ping me because I have to confirm you. I havenâÂÂt figured out all the settings so hopefully this will change as well soon. 19:31:40 <fjahr> That's already it, I wasn't present at the last core dev but I read the notes on the topic so I thought it might be helpful 19:32:06 <MacroFake> fjahr: Would be good to answer the basic questions from the wiki 19:32:09 <achow101> we have a wiki page with some links to older discussions on this topic: https://github.com/bitcoin-core/bitcoin-devwiki/wiki/GitHub-alternatives-for-Bitcoin-Core 19:32:40 <MacroFake> https://github.com/bitcoin-core/bitcoin-devwiki/wiki/GitHub-alternatives-for-Bitcoin-Core#evaluation-scheme 19:33:11 <fjahr> right, yeah, so if anyone wants to help me answer these questions, let me know :D 19:33:12 <jon_atack> achow101: i think the idea was to include the ASMap data in the distributed binary but not in the repository. discussion about this took place here: https://www.erisian.com.au/bitcoin-core-dev/log-2019-06-20.html#l-632 19:33:18 <jon_atack> (sorry, catching up) 19:35:21 <achow101> any other topics to discuss? 19:35:51 <jon_atack> achow101: mind adding https://github.com/bitcoin/bitcoin/pull/26283 to the blockers list? 19:36:06 <jon_atack> #26283 19:36:09 <gribble> https://github.com/bitcoin/bitcoin/issues/26283 | p2p: Fill reconciliation sets and request reconciliation (Erlay) by naumenkogs ÷ Pull Request #26283 ÷ bitcoin/bitcoin ÷ GitHub 19:36:31 <jon_atack> seems to be in final review phase, i plan to get to it 19:36:36 <achow101> jon_atack: done 19:36:49 <jon_atack> achow101: ty 19:36:53 <achow101> #endmeeting