Logs:Minimally Catastrophic

Minimally Catastrophic
Dramatis Personae Bruce, Kisha
In Absentia	2019-07-25 "And I'd love to check out something that might advance science or blow up the city."
Location <NYC> Stark Tower - Midtown East
The R&D division canteen doesn't have the same selection as the main employee canteen, focusing mainly on food people can pick up in a hurry and take back to their desks, but it does have some perks. You'll likely never have to struggle to find a table even during the lunchtime rush. Kisha is usually in the 'food at her desk' club, but today there are moderately hazardous chemicals in the robotics lab. So she's actually taking the time to eat a full and nutritionally balanced meal. She's wearing her general 'office casual' oufit. A randomly selected t-shirt with a joke printed on it in binary and black pants finished with a labcoat on over the top that has all her various work ID badges clipped to it. Of course the eating in the canteen part is falling down on the 'eating' part. Her attention more taken up with a big pile of technical documents that she's flicking through and scribbling out various lines, underlining others and every now and again scribbling notes in the margins. Bruce slinks into the canteen, looking like he hasn't slept. His mop of black hair hangs unruly, his thick-framed glasses sit crooked on his nose, and there are deep shadows beneath his eyes. He's wearing a forest green dress shirt, no tie, and camel trousers under his (also badge-ridden) lab coat. He cradles a large, thin tablet in the crook of one arm and is tapping at its screen with a look of deep dissatisfaction on his face. He looks up from his work long enough to fetch himself a cappuccino and a tempeh foccacia sandwich (using his tablet for a tray), and looks about to ready to leave when he spots Kisha and detours toward her. "Hey, um." He come to a stop, looks down at his lunch, and back to his coworker. "If you're busy I can just -- send you an email, but I was wondering if you have a moment and wouldn't mind helping me with something?" Kisha doesn't initially react to the question. Then a few beats later she glances around and frowns. "Oh, you meant me?" she wonders. "I have time. What is it you need? Something we can discuss here or...?" She flips a few more pages and then frowns at her document and closes it. "I'm not really achieving anything important right now. Document revision." Bruce sets down his lunch, removing them from the tablet and then swiping something out on its screen while he speaks. "Oh yeah, here is fine. I've been having technical difficulties with the particle accelerator design I'm working on and I'm beginning to think the entire engineering /perspective/ is wrong. I am not," he clarifies, "an engineer myself, and the ones on my team have been following my lead too much, I think." He activates a small holographic projection of a complex 3D diagram. At its base it looks like a rosette of emitters all aimed at a central cylinder. "The issue is I need dynamic adjustments on the energy input, and pre-loaded equations aren't cutting it." He turns both hands palm-up in resignation. "Since you work with adaptive algorithms I thought you might have some insight on whether this might be a good fit for a task-specific AI." Kisha shifts on her seat and tucks one of her legs under herself. Perching awkwardly on the chair. "The trouble with AI is training it to do what you want in a consistent way," she muses. "Personally I try keep my software as dumb as possible and then handle all the adjustments myself. But that's not really ideal if you're wanting to focus on the results or doing high energy reactions that take fractions of a second." She taps her pen a few times against the table. "But we've certainly got machine learning capable of handling the task. The project I'm working on... We use a system that responds to the slightest changes in brainwave patterns. Then makes appropriate adjustments to the hardware." Bruce nods eagerly. "That's exactly the issue--no operator could conceivably react fast enough to make the needed adjustments." He frowns, suddenly thoughtful. "Well, no, but the number of people out there who could react fast enough is small enough to render the prospect impractical. It's just..." He rubs his thumb over the rubberized edge of the tablet. "Well. My understanding of machine learning is fairly basic, and I'm concerned about unpredictable behavior. Of course the system would be trained in simulation, but simulations can never fully account for the range of possible variables. If something comes up that the system has no experience handling, is there some way to guide its response to something..." He hums quietly. "... minimally catastrophic?" After twirling the pen a few times Kisha begins doodling a few numbers on the front of her papers. "It's all a question of how you train it," she eventually replies. "If your process is likely to have chances of catastrophic failures that could be predicted... Well that's pretty easy." She bites the end of her pen, frowns, then starts scribbling a little diagram. "Right so think about it like this. You set the experiment up like a game. The AI gets a score for everything it does. It knows a high score is better and, providing it's set up right, will aim to get the best possible result." "Then you give every possible type of failure a points value. Blow up the city gets zero points. Blow up the building maybe one. While just the lab is two. That way the AI will always try to keep the scale of any incident as small as possible.... It's not fool proof though. If your training models don't match up with reality the software might think it's trying to keep casualties small when it's actually killing everyone in a five block radius." Bruce leans forward and studies the diagram. "Yeah, see the problem is I'm just not confident we can conceivably supply every possible failure condition. Maybe not even every /probable/ one." He traces a finger over the ring of field emitters in his projection, highlighting them. "To my knowledge, nobody has ever tried to build a linear accelerator with a circular arrangement of field inducers before. So most of the information that we would be able to supply for training and algorithm are theoretical. I suppose we could instruct it to initiate a process shutdown if it encounters anything outside of its experience?" "I can't think of any reason why that wouldn't work. Just so long as there are no hypothetical situations in which a sudden shut down could cause an even worse disaster?" Kisha wonders, scribbling a few notes next to her diagram. "What I can do is... once I'm back at my workstation I'll forward you all the contacts I've consulted with about machine learning and AI systems. If that sounds like something you'd find helpful?" "I can't be sure they'll all have the time to help, but I expect they'll be happy to at least to recommend software to use as a base. Maybe suggest some research papers you'd like to read. Depends on if your project has resources to officially book their time." "Not a /worse/ disaster, no." The degree to which Bruce sounds speculative on this point is perhaps not particularly reassuring, but he doesn't seem discouraged. "I'll get my actual engineers to sort out the details--I just wanted the opinion of someone who has no vested interest in staying on my good side. But, a list of contacts would still be much appreciated, when you can spare a few minutes." He pulls his glasses off and minimizes the projection. "I appreciate your input, though. I'll let you know when we get it up and running, in case you want to come by for a look." He stands up and gathers his lunch, frowning. "Or...you know. Take a day off." Kisha laughs. "It's no trouble," she replies with a nod. "And I'd love to check out something that might advance science or blow up the city. Especially as I don't think my apartment is far enough away to actually be safe in the event of a serious disaster.." "Oh no, don't worry," Bruce says hastily, waving his hands. "If something went wrong it'd likely just make a mess of the testing chamber." He pauses, considering. "Alright if it went /really/ wrong it might be bad news for the whole floor floor, but I have faith in this building's integrity." He turns his tablet into a tray again. "Have a lovely day!" "Just make sure nothing else on the floor can explode," Kisha suggests with a nod. "You never know just what doomsday weapons someone might be cooking up in this place." She raises the still sealed bottle of fruit juice from her meal and tips it in a vague farewell. "You too."