16:00:11 <stickies-v> #startmeeting 
16:00:11 <corebot> stickies-v: Meeting started at 2026-08-27T16:00+0000
16:00:12 <corebot> stickies-v: Current chairs: stickies-v
16:00:13 <corebot> stickies-v: Useful commands: #action #info #idea #link #topic #motion #vote #close #endmeeting
16:00:14 <corebot> stickies-v: See also: https://hcoop-meetbot.readthedocs.io/en/stable/
16:00:15 <eugenesiegel> hi
16:00:16 <corebot> stickies-v: Participants should now identify themselves with '#here' or with an alias like '#here FirstLast'
16:00:19 <yuvicc> hi
16:00:20 <sedited> hi
16:00:21 <dergoegge> hi
16:00:22 <pseudoramdom> hi
16:00:22 <janb84> hi
16:00:22 <andrewtoth> hi
16:00:24 <lightlike> hi
16:00:24 <stickies-v> #bitcoin -core-dev Meeting: _aj_ abubakarsadiq achow101 andrewtoth b10c brunoerg cfields danielabrozzoni darosior dergoegge dzxzg eugenesiegel fanquake fjahr furszy hebasto hodlinator instagibbs janb84 jarolrod johnny9dev jonatack josie jurraca kanzure kevkevin laanwj lightlike l0rinc maflcko marcofleon maxedw Murch pinheadmz provoostenator pseudoramdom ryanofsky sdaftuar sedited sipa sliv3r__ sr_gi stickies-v
16:00:24 <stickies-v> stringintech theStack vasild willcl-ark
16:00:25 <Moneyball> hi
16:00:29 <johnny9dev> hi
16:00:30 <danielabrozzoni> hi
16:00:30 <hebasto> hi
16:00:34 <tigerMafia> hi
16:00:40 <stringintech> hi
16:00:48 <nervana21> hi
16:00:50 <pinheadmz> Yo
16:00:59 <l0rinc> hi
16:01:05 <yancy> hi
16:01:48 <stickies-v> There is one pre-proposed meeting topics this week. Any last minute ones to add?
16:01:58 <dergoegge> Steve had one above
16:02:10 <dergoegge> "Bitcoin Core and AI usage, perception and reality"
16:02:18 <stickies-v> yup, that's the "one", ty
16:02:22 <dzxzg> hi
16:02:42 <dergoegge> oh sorry misread the message
16:02:50 <stickies-v> #topic Fuzzing WG Update (dergoegge, marcofleon)
16:03:15 <dergoegge> no update, but i did want to mention that this is a cool PR: https://github.com/bitcoin/bitcoin/pull/36098
16:03:23 <maxedw> hi
16:03:56 <stickies-v> #topic Kernel WG Update (sedited)
16:03:58 <sedited> nothing from me
16:05:01 <stickies-v> #topic Benchmarking WG Update (l0rinc, andrewtoth)
16:05:03 <l0rinc> Recently I've been focusing on finding accidental quadratic algos throughout the codebase (after bumping into one in #35889), had a few good findings
16:05:06 <corebot> https://github.com/bitcoin/bitcoin/issues/35889 | rpc: avoid quadratic `gettxspendingprevout` work and preserve order by l0rinc · Pull Request #35889 · bitcoin/bitcoin · GitHub
16:05:23 <l0rinc> rebasing and reviewing similar findings, that's it from me
16:05:49 <andrewtoth> nothing from me
16:06:29 <stickies-v> #topic QML GUI WG Update (johnny9dev)
16:06:48 <johnny9dev> We have a new contributor to the project, uqlidi, who has been helping fix up bugs and compatibility issues with the qml. pseudoramdom has been doing a lot of good work addressing feedback on the preview release and making really nice design updates to all of the views. epicleafies has also been addressing feedback on the Acitivity page
16:07:00 <johnny9dev> I made a good number of updates to the first staging branch pr (bitcoin-core/gui-qml#871)
16:07:01 <corebot> https://github.com/bitcoin-core/gui-qml/issues/871 | Staging Initial Foundational commits by johnny9 · Pull Request #871 · bitcoin-core/gui-qml · GitHub
16:07:16 <Murch[m]> hi
16:07:22 <achow101> hi
16:07:29 <johnny9dev> as a reminder, the staging branch is what we hope to merge into bitcoin/bitcoin
16:08:38 <johnny9dev> I got some really good feedback on it already and have shifted some things around. One of the bigger changes will be balancing our test automation strategy by including an integration suite that run QtTest to test the Gui+Node in process
16:08:53 <hebasto> asking for more eyes in qml#881 as it paves the way for further work on the staging branch
16:08:54 <corebot> hebasto: Error: That URL raised <HTTP Error 404: Not Found>
16:08:58 <Murch[m]> It’s great to see the progress y’all are making. Maybe we can do another testing party when you think it’s in a good place for that
16:09:05 <johnny9dev> this will let us move a lot of our end to end functional regression tests into a c++ qttest target
16:09:17 <johnny9dev> 871*
16:09:54 <hebasto> ^ yes, https://github.com/bitcoin-core/gui-qml/pull/871
16:10:17 <johnny9dev> I'm iterating on the description to try to make it clear what the decisions currently made are and I hope that the current description should at least make it clear the main things the PR does to create the initial foundation
16:11:26 <hodlinator> hi
16:11:40 <jonatack> hi
16:12:21 <johnny9dev> but yeah looking for feedback and opinions on the staging PR now as it will have a significant impact on the staging going forward.
16:12:44 <johnny9dev> and everything learned from the foundation staging PR i will be working to add to our development qt6 branch
16:12:53 <johnny9dev> that is all
16:14:33 <stickies-v> #topic Bitcoin Core and AI usage, perception and reality (Moneyball)
16:14:40 <Moneyball> Several people, including donors have approached me expressing concern that Bitcoin Core devs "aren't paranoid enough about AI cyber threats" and that the Core project isn't sufficiently leveraging AI tools for tasks like assisting issue and PR triage, code review, and test case development.
16:14:47 <Moneyball> I've spoken to a few Core devs, so I am aware of some AI activity and usage within Core, but to be honest I don't have a clear sense of the extent of its use. I sense that it is used more than the current perception suggests. It is also my sense that its usage could increase further. In any case, I don't really have anything substantial to share with folks.
16:14:52 <Moneyball> Thoughts on publishing more about how individuals or the project as a whole is utilizing AI for offense and defense?
16:17:04 <pinheadmz> We're all using Claude lol
16:17:55 <pinheadmz> There's a few engineers like furszy building their own harness and scanning for things 24/7 or however many tokens he gets per week
16:18:07 <pinheadmz> Not to mention Rob Ham and "red team"
16:18:35 <achow101> plenty of people are using ai for assistance. but just because it exists also does not mean that everyone needs to use it
16:18:40 <yuvicc> project loupe as well ig
16:18:55 <stickies-v> I use LLMs quite heavily for additional review. I'll do my own manual review, have an LLM do its own parallel pass, and then combine both / dive deeper from there
16:19:01 <achow101> and frankly, ai use has produced so much noise and garbage that i'm not interested in seeing more from people who don't know what they're doing
16:19:22 <Moneyball> I think that attitude is the problem
16:19:36 <stickies-v> but yeah I prefer people owning their reviews instead of fully automated AI reports which seem to be very very noisy on our codebase
16:19:37 <Moneyball> I'm well aware of where AI is and is not valuable.
16:19:45 <eugenesiegel> I am not involved in red team or project loupe or anything and i can't speak for the project as a whole, but I don't use claude that much except for dumb scripts. I also think a lot more effort could be spent training people to actually spot bug classes instead of relying so much on AI but that's probably a whole conversation. I'd also be worried
16:19:46 <eugenesiegel> about this encouraging or forcing contributors to use AI and I can think of another project that uses it really heavily and I honestly think the quality of the codebase has gone down since. Probably have more opinions that can't fit in here, and I'm not an ai doomer or anything...
16:20:22 <Murch[m]> My impression is that many or most of the Bitcoin Core contributors have started incorporating AI tooling into their workflows and given their prior expertise get great use out of it.
16:21:19 <Moneyball> My high level point is "how does the bitcoin ecosystem know what the Core project is and is not doing with respect to AI in order to reassure folks that the contributors to the project aren't sticking their heads in the sand"
16:21:28 <dzxzg> So a broad spectrum of views and ways that people are using it
16:21:45 <Moneyball> I'm here to help. I'm aware of several efforts. Not all. But I don't really know what to tell people who ask.
16:21:52 <achow101> The "project" is not using it in any way. individuals are using ai in a variety of ways
16:21:55 <sedited> Moneyball, I'm not sure where the impression of sticking their heads in the send comes from.
16:21:57 <jonatack> Moneyball: from where do these folks get their impressions
16:21:59 <eugenesiegel> I kind of equate ai scanning to something like sqlmap where you scan for low hanging fruit (i.e. common bug classes like reading past end of buffer, etc) and more complicated bugs are probably not going to be found with it (yet).
16:22:06 <andrewtoth> I use AI tools for absolutely everything. I don't think I can go back to not being able to use it. I never post text I expect other humans to read from AI though, I always rewrite in my own words so I am sure I understand what I'm writing.
16:22:14 <Moneyball> yes achow101 that is precisely why i said contributors
16:22:21 <Moneyball> as i knew you'd have snide remarks
16:22:37 <sedited> We're frustrated with slop contributions, yes, but I think many of us are using the tools pretty effectively.
16:23:04 <andrewtoth> eugenesiegel: I see the opposite. It's finding obscure bugs that would be very difficult for humans to find.
16:23:27 <Moneyball> jonatack i'm not sure where people get their impressions but wherever they search or look i don't think they'd find the answer to the question
16:23:50 <eugenesiegel> andrewtoth: where? in bitcoin core or somewhere else?
16:24:40 <andrewtoth> clightning is one example, I haven't seen the bugs that were found though
16:24:42 <johnny9dev> What can be done even? As these things seem to get better it doesnt look like anything other than people being even more productive?
16:24:46 <Moneyball> one approach that project goose is trying to deal with slop is https://goose-docs.ai/blog/2026/07/30/issues-are-the-new-prs/
16:25:44 <Moneyball> a group of us are meeting in a couple of weeks to discuss learnings from Loupe and red team, how projects can handle triage of these reports, and we will be publishing best practices. this will include l0rinc and justin from Localhost
16:25:47 <lightlike> there are different ways: a few people use AI to find ideas for issues/PRs (I don't), more people (including me) use it as a tool in PRs / issues they've come up with by themselves.
16:26:43 <eugenesiegel> andrewtoth: I haven't really seen an example of something that's so complicated a human couldn't find it, but if I'm sticking my head in the sand I'd like to know. May be anecdotal, but I haven't seen nearly the same amount of attention given to something like syzkaller in the past (linux kernel fuzzer) compared to the attention AI is getting now
16:27:05 <andrewtoth> what about the zcash inflation bug?
16:27:16 <Moneyball> eugenesiegal i can definitely state that Loupe has found many, many vulns across dozens of projects. some are high severity. many are medium severity but these can be chained together to create a high severity outcome
16:27:56 <eugenesiegel> andrewtoth: I don't know much about the zcash bug, so I'm probably out of the loop
16:28:44 <johnny9dev> Even if a human can find it the speed and thoroughness of the frontier models are undeniable
16:28:49 <andrewtoth> Moneyball: if the AI can't chain the medium severity bugs itself to produce the high severity, then I find these claims kind of hand-wavy
16:29:31 <andrewtoth> eugenesiegel: https://forum.zcashcommunity.com/t/the-orchard-counterfeiting-vulnerability-and-next-steps/56015
16:29:45 <eugenesiegel> Moneyball: speaking from experience, some of the codebases I've seen in the space are really lacking on the security side and are missing really basic checks. but i'll wait for public disclosure to judge for myself
16:29:48 <sedited> Moneyball, "issues as PRs" has not worked well here so far. Many bots do take things off the issue tracker, but a lot of them produce slop that then needs more triaging and work from more people. Take this PR from today for example, which took away attention from a bunch of us, but didn't do anything to move things forward: https://github.com/bitcoin/bitcoin/pull/36055 . Can obviously say
16:29:50 <sedited> that the problem is the issue tracker, but would take significant reform to change.
16:30:04 <dergoegge> Personally I think core has been holding up well against the "AI threat" because we have very strong testing for most important things, that other project like e.g. some of the lightning impls have not invested in the prior years
16:30:24 <sedited> dergoegge +1
16:30:29 <jonatack> I use free AI sometimes to verify things. I don't use it at all (for now) for writing. I would like to ramp up on open source models but in El Salvador it is hard to obtain hardware without importing it yourself. Frontier token use is a cost I believe I can't take on alone.
16:30:33 <jonatack> dergoegge: agree
16:30:47 <johnny9dev> yes bitcoin's strong engineering fundamentals are paying off
16:31:07 <dergoegge> That said I do think AI vuln scanning is obviously a part of any project that wants to take security seriously
16:31:22 <jonatack> yes
16:31:43 <Moneyball> sedited i'd be happy to engage you and interested folks on exploring how to handle AI noise. can follow up after
16:31:45 <l0rinc> I started scanning for bugs and either fixing them or reporting them to security list (to Core and a few other projects) for the past few months. Unfortunately this is often costing me thousands of dollars per day so far but OpenSats just approved a mini-grant that I could use to continue these scans.
16:32:51 <l0rinc> I understand that some people think the findings are garbage - which was the case up until a few months ago -, but that's not the case anymore, basically all of the findings are real now, but without the cyber capabilities only the low-severity ones gets surfaced.
16:33:09 <eugenesiegel> andrewtoth: thanks for the link. I just don't know how to evaluate this, so I don't really know how complicated it is. Though there's something to be said that it wasn't found prior and was only found via AI
16:33:24 <achow101> Moneyball: I don't understand what the concern in your original question is. Do people think that ai is not being used?
16:33:26 <Moneyball> also the question of "how is Core utilizing AI" isn't just about AI security scanning but also other use cases of AI
16:34:00 <Moneyball> achow101 people have a belief the Core project is behind the curve. I don't have anything to point them to that counters that view
16:34:40 <tigerMafia> Moneyball: about mindless AI PRs (slop) - i agree with achow101, if someone is does not know what they're doing, LLMs will just amplify that skill issue a lot more and make it a pain for the reviewers. personally, i have been faced with fully AI PRs where authors bank on reviewers to steer their harness (not so good). if someone copy-pastes AI output, bit for bit, i ignore the PR.
16:34:49 <maxedw> I got to see Furzy's harness in action and it was quite impressive, I know he has been scanning the codebase for months. If external people want to help in those efforts he could probably massively benefit from access to the most capable model without the security classifiers.
16:35:25 <Moneyball> tigerMafia i fully appreciate that and sympathize with you. we're seeing it across many projects and as mentioned there are various attempts to combating it.
16:37:06 <pseudoramdom> How are other projects giving the perception of "using AI" apart from just scanning issues?
16:37:15 <Murch[m]> My take away is that many contributors have been using AI, the project has stood up well to recent red team probing, but AI tools have not been incorporated at the project level yet. Presumably the absence of the latter is what prompts the concern
16:37:16 <dergoegge> Not to jinx it but, maybe the fact that the bitcoin network is still up and running, is a hint that core is not behind on things?
16:37:21 <jonatack> Moneyball: AI has been (over-)used to prompt finished-looking, verbose, often nonsensical, techno-babbly BIP drafts by nyms that (often) open them directly as PRs to the BIPs repo. OTOH, it is being used well by established bitcoin protocol devs to propose useful BIPs.
16:37:36 <sedited> sounds like the problem is more that people are not blogging enough :P
16:37:46 <Murch[m]> pseudoramdom: E.g., automatic review of new issues and PRs by agents
16:37:58 <achow101> perhaps people are more interested in doing useful things with ai rathe than talking about it
16:38:16 <andrewtoth> I would not be happy if we had a copilot-like bot that posted automatic review comments, if that is an idea someone would have
16:39:08 <l0rinc> andrewtoth: copilot is not very smart, but making drahtbot smarter to find non-surface level findings would be very useful
16:39:16 <yancy> Perhaps part of the perception issue is also that core is a C++ code base, and AI is not maybe as effective at writing C++ as it is Rust due to the type system.  Just speculating why perception is different from the reality.
16:39:22 <johnny9dev> It does almost feel too early to add anything additional to bitcoin core. On top of it, bitcoin core is likely extremely well understood already by llms so is there any markdown file that can even really add any additional value to this?
16:39:34 <l0rinc> yancy: LLMs are very proficient in C++ now
16:39:40 <johnny9dev> codex is very good at modern c++
16:39:48 <andrewtoth> l0rinc: I don't want automatic comments from bots. If a reviewer uses a smart AI and then triages the results in their own words, ensuring no false positives, that would be welcome.
16:39:48 <tigerMafia> andrewtoth: +1
16:40:13 <lightlike> agree, AI during review is most helpful if you can interact with it / steer it into directions where real issues may be. an automatic AI review isn't helpful, everyone can do a rough analysis locally and then take it from there.
16:40:21 <andrewtoth> I can run my own AI reviewer myself, I don't need it posting on my PR
16:40:47 <l0rinc> andrewtoth: I sympathise with the feeling, but some PRs have so many bugs that I would prefer an AI point them out so by the time we get to it the trivial ones are already handled
16:41:00 <andrewtoth> no
16:41:06 <l0rinc> yes :)
16:41:23 <eugenesiegel> are there certain areas that LLMs are lacking in or are security engineers cooked?
16:41:33 <achow101> i absolutely do not want automated review
16:42:23 <maxedw> also not a fan of automated review, I use privately and find 4/5 complaints are non-issues or random opinion
16:42:24 <tigerMafia> andrewtoth: +1. already github has an issue with handling comments.. now imagine the volume of comments that would be made by copilot
16:43:01 <Moneyball> eugenesiegel there is still value in the human driving the LLM to find security vulns. being a subject matter expert helps.
16:44:01 <l0rinc> I also don't like spammy comments, but if configured well, an automated AI review would be extremely useful. We all have PTSD from last year's models which were mostly noise: that's not the case anymore.
16:45:02 <Murch[m]> Well, seeing the garbage we get in Optech and the BIPs repository, it doesn’t seem everyone has stopped using last year’s models
16:45:13 <jonatack> :D
16:45:17 <_aj_> have individual contributors run automatied AI reviews, read/understand them, then pass on the key findings without huge chunks of AI claude-ish verbiage
16:45:19 <eugenesiegel> do people have certain prompts they use for this? Even when I ask claude benign security-related questions, it refuses to respond. And I haven't gotten access to the chinese models yet :/
16:45:57 <tigerMafia> eugenesiegel: chinese models tend to respond, no qualms
16:46:22 <andrewtoth> _aj_: +1
16:47:28 <janb84> _aj_: for now the best strategy I think
16:47:39 <l0rinc> _aj_: absolutely, but that's a lot of work, it shouldn't be the only way to receive feedback - this way I personally just ignore reviewing PRs that aren't structured well. Maybe we could enable AI review for new contributors only and see how well it performs.
16:47:41 <fanquake> _aj_: mixed output on that front. I think we still need to push for better understanding of issues before they are being forwarded, along with the production of accompanying (functional) tests
16:48:33 <andrewtoth> the "read/understand them" part is the hardest
16:48:58 <andrewtoth> otherwise you offload that onto the author, rather than that being the reviewer's job
16:49:35 <l0rinc> if the AI only receives a patch (without any tool calling and full repo access) we can't expect depth. But a proper review bot that checks out the change and does its thing is extremely capable lately!
16:50:00 <willcl-ark> people are worried we don't have enough AI in the Github repo?
16:50:08 <tigerMafia> :)
16:50:09 <andrewtoth> l0rinc: we can all do that locally though, no need to pollute github with it
16:50:35 <l0rinc> we can, but most people aren't and we're commenting on stuff that an AI could have found
16:50:46 <Moneyball> willcl-ark not sure if that is a joke or actual question
16:51:08 <achow101> l0rinc: so?
16:51:18 <willcl-ark> Moneyball: I'm catching up from "donors have approached me expressing concern that Bitcoin Core devs "aren't paranoid enough about AI cyber threats" and that the Core project isn't sufficiently leveraging AI tools for tasks like assisting issue and PR triage, code review, and test case development."
16:51:48 <l0rinc> ahow101: so if human review is the bottleneck, let's use AIs to help with the low hanging comments
16:51:52 <achow101> ai found an issue, human finds an issue, same result, issue is found and fixed
16:52:26 <achow101> like if i'm reviewing a pr, and someone else reviews the same pr, and we both find the same issue, that's not a problem. don't see why it would be any different if an ai did it or not
16:53:04 <l0rinc> achow101: no, it's like proofreading a document and you find typos - a spell checker should have caught those instead of wasting human reviewer efforts
16:54:30 <_aj_> l0rinc: same applies to a linter or running CI; in this case the idea is that they're not quite reliable enough to run as part of CI on every PR, so you should run them yourself as an author or reviewer
16:54:36 <achow101> i don't think it's wasting, but i get your point
16:55:00 <eugenesiegel> do people have any tips to make the output of the llms less noisy or to focus on things that are actionable?
16:55:23 <Rob1Ham_> Hello everyone, first time using IRC in 20 years, I was told to join, happy to help however I can.
16:55:28 <willcl-ark> AFAIK ~ everyone is using AI tooling locally to assist with coding, review, building tooling, benchmarking, test case development, fuzzing, whatever else they're working on. The only thing "missing" for these worried people therefore is the "lack" of automated bot review in Github?
16:55:39 <johnny9dev> eugenesiegel: not joking, tell it thats what you want
16:56:07 <l0rinc> _aj_: "they're not quite reliable enough to run as part of CI on every PR" - I'd argue they absolutely are reliable enough now, given proper prompting and context and price
16:56:28 <_aj_> l0rinc: other people don't agree, and arguing isn't going to convince them
16:56:31 <Murch[m]> willcl-ark: That seems to be the takeaway
16:56:32 <Moneyball> willcl-ark no one said anything about automated bot review in github as what is lacking in public communications
16:56:34 <jonatack> willcl-ark: could also use hardware or token budgets, but maybe that's only me
16:56:49 <Moneyball> murch how is that the takeaway? it seems made up? i never said that
16:57:00 <andrewtoth> Can we just have a guideline to run your code through an LLM before posting? Nobody wants to read the output of an LLM on your PR, they want humans to write and respond to.
16:57:06 <willcl-ark> OK perhaps I'm missing what is being looked for here. Publishing which tools we use and how?
16:57:08 <johnny9dev> can't someone just go and try to secretly prove out an AI review bot?
16:57:18 <Murch[m]> “the Core project isn't sufficiently leveraging AI tools for tasks like assisting issue and PR triage, code review, and test case development.”
16:57:28 <sipa> i'm not sure what the point of this discussion is
16:58:00 <Moneyball> murch that statement doesn't imply or state that auto review bots in github is required?
16:58:24 <Murch[m]> Well, if you have a different take away, how about you share yours
16:58:24 <jonatack> Moneyball: you would like info to point to, that can reassure worried people IIUC
16:58:39 <willcl-ark> jonatack: I mean, I wouldn't be opposed to a dev token fund; then people could see how many token we were using!
16:58:50 <stickies-v> we're also coming up on time. I suggest we wrap this up and if there are follow up specifics to be discussed after today's initial discussion, we can do that next week?
16:58:54 <dergoegge> I think the takeaway is that we should be communicating our work more (which we discussed at prior coredevs as well)
16:59:00 <lightlike> Moneyball: then what do you suggest should change from the status quo, where already almost everyone  uses AI locally, with a few projects doing systematic bug finding? What would be "state of the art" if the status quo isnt?
16:59:05 <sipa> Moneyball: are you asking what people are using? trying to encourage some kind of usage? better documenting?
16:59:12 <Moneyball> sipa i was asked to come here to raise this topic. i'm merely asking "what content can be shared with inquiring minds about how the Core project isn't behind the times on utilizing AI." i have never brought up the topic of "you must use a review bot in github" that is made up.
16:59:46 <andrewtoth> that was me bringing up it was a bad idea, sorry for the noise
16:59:54 <Moneyball> andrewtoth no worries
17:00:09 <willcl-ark> I think that's the main way one would see issue and pr triage/review taking place in the open by AI
17:00:16 <achow101> Moneyball: then this discussion should suffice as public documentation that it's happening
17:00:45 <Rob1Ham> I have plenty of thoughts on the topic, but don't wish to further derail the conversation if it is out of scope for this discussion. I'm happy to share my perspective at another time or individually, anyone is welcome to reach out to me anytime.
17:01:21 <Murch[m]> Moneyball: your initial topic prompt specifically stated that people had the impression that we were behind the curve on AI, because “the Core project isn't sufficiently leveraging AI tools for tasks like assisting issue and PR triage, code review, and test case development.”, so it’s confusing to me that you push back on me concluding that someone looked at our repository and found it strange that we don’t have automated PR review.
17:01:24 <jonatack> Rob1Ham: after the meeting can work, too, if people aren't against
17:01:41 <BlueMatt> speaking on the LDK end, where we have a project that takes security very seriously but is also a rather complicated set of code - we get very high value with having a cheap opus review bot do a first pass on PRs. It regularly finds real issues, and that's a *really* dumb bot, people doing manual review passes with better LLMs obviously finds much more. On the security side, LLMs are better at finding almost any class of bug than humans at
17:01:41 <BlueMatt> this point, there's almost no question, and if its not true give it about two months.
17:02:04 <Moneyball> achow101 i can summarize this discussion and post about it or speak about it. i'd have many more detailed questions though. i could write these questions down and share. then if any Core dev is so inclined, you could answer them in a post or here or privately with me and let me know what you are comfortable with sharing publicly
17:02:06 <BlueMatt[m]> there's a lot of value in being a subject-matter expert and driving it in the right direction
17:02:13 <Rob1Ham> My very brief summary is that the past month, of using either the frontier labs with cyber approved access, or the most recent open source models have seen a considerable step function in capability in finding vulnerabilites. I've used these tools for years and the past few months is a serious step fucntion.
17:02:26 <l0rinc> BlueMatt: +1
17:02:52 <Moneyball> murch sorry i just don't see how my statement in any way asserts auto bot github activity. it is made up.
17:03:11 <BlueMatt[m]> LLMs are also at the point where I have very little desire to review human-authored code - LLMs are just better.
17:03:15 <dergoegge> I had a similar discussion with Steve somewhere else, and suggested that he bring this up here. Specifically the public comms aspect.
17:03:15 <dergoegge> Even though this may have derailed at the end here, I think there were some good points being made, and I'm sure this discussion can continue at a future coredev
17:03:15 <dergoegge> Thank you, moneyball!
17:03:28 <achow101> Moneyball: please do that
17:03:35 <BlueMatt[m]> of course I also have no desire to review purely LLM-authored code where a human wasn't involved in taking a lot of care with how the code gets structured
17:03:37 <Moneyball> achow101 will do. thank you.
17:04:12 <l0rinc> moneyball: thanks for bringing up the topic
17:04:53 <stickies-v> thanks for the discussion everyone, let's wrap it up here
17:05:00 <stickies-v> #endmeeting