https://www.amazon.science/blog/automated-reasoning-at-federated-logic-conference-floc * Research areas + Automated reasoning + Cloud and systems + Computer vision + Conversational AI / Natural-language processing + Economics + Information and knowledge management + Machine learning + Operations research and optimization + Quantum technologies + Robotics + Search and information retrieval + Security, privacy, and abuse prevention + Sustainability + Automated reasoning + Cloud and systems + Computer vision + Conversational AI / Natural-language processing + Economics + Information and knowledge management + Machine learning + Operations research and optimization + Quantum technologies + Robotics + Search and information retrieval + Security, privacy, and abuse prevention + Sustainability * Blog * News and features + Awards and recognitions + Awards and recognitions * Publications * Conferences * Collaborations + Academics at Amazon + Alexa Prize + Amazon Research Awards + Amazon SURE + Academics at Amazon + Alexa Prize + Amazon Research Awards + Amazon SURE * Careers + Internships + Working at Amazon + Internships + Working at Amazon [] Feedback Follow Us * twitter * instagram * youtube * facebook * linkedin Menu amazon-science-logo.svg * Research areas + Automated reasoning + Cloud and systems + Computer vision + Conversational AI / Natural-language processing + Economics + Information and knowledge management + Machine learning + Operations research and optimization + Quantum technologies + Robotics + Search and information retrieval + Security, privacy, and abuse prevention + Sustainability + Automated reasoning + Cloud and systems + Computer vision + Conversational AI / Natural-language processing + Economics + Information and knowledge management + Machine learning + Operations research and optimization + Quantum technologies + Robotics + Search and information retrieval + Security, privacy, and abuse prevention + Sustainability * Blog * News and features + Awards and recognitions + Awards and recognitions * Publications * Conferences * Collaborations + Academics at Amazon + Alexa Prize + Amazon Research Awards + Amazon SURE + Academics at Amazon + Alexa Prize + Amazon Research Awards + Amazon SURE * Careers + Internships + Working at Amazon + Internships + Working at Amazon [] Feedback Search [ ] Submit Search Automated reasoning Automated reasoning at Amazon: a conversation To mark the occasion of the eighth Federated Logic Conference (FloC), Amazon's Byron Cook, Daniel Kroning, and Marijn Heule discussed automated reasoning's prospects. By Larry Hardesty August 08, 2022 Share Share * Copy link * Email * Twitter * LinkedIn * Facebook * WhatsApp * Reddit * QZone * Sina Weibo Conference * FLoC 2022 Related publications * Migrating solver state The Federated Logic Conference (FLoC) is a superconference that, like the Olympics, happens every four years. FLoC draws together 12 distinct conferences on logic-related topics, most of which meet annually. The individual conferences have their own invited speakers, but FLoC as a whole has several plenary speakers as well. At the last FLoC, in 2018, one of those plenary speakers was Byron Cook, who leads Amazon's automated-reasoning group, and he was introduced by Daniel Kroning, then a professor of computer science at the University of Oxford Byron Cook's keynote at FLoC 2018 With introduction by Daniel Kroning. "What makes me so proud that Byron is here," Kroning said, is "he's now at Amazon, and he's going to run the next Bell Labs, he's going to run the next Microsoft Research, from within Amazon. My prediction is that -- not 10 years but 16 years; remember, it's multiples of four -- 16 years from now you'll be at a FLoC, and you'll hear these stories about the great thing that Byron Cook built up at Amazon Web Services. And we'll speak about it in the same tone as we're now talking about Bell Labs and Microsoft Research." In the audience at the talk was Marijn Heule, a highly cited automated-reasoning researcher who was then at the University of Texas. "I hadn't met Marijn, though I had heard about him from a couple other people and thought I should talk to him," Cook says. "And then Marijn found me at the banquet after the talk and was like, 'I want a job.'" AR scientists.png L to R: Amazon vice president and distinguished scientist Byron Cook; Amazon Scholar Marijn Heule; Amazon senior principal scientist Daniel Kroning. Heule is now an Amazon Scholar who divides his time between Amazon and his new appointment at Carnegie Mellon University. Kroning, too, has joined Amazon as a senior principal scientist, working closely with Cook's group. As 2022's FLoC approached, Cook, Kroning, and Heule took some time to talk with Amazon Science about the current state of automated-reasoning research and its implications for Amazon customers. Policy-code.gif Related content A gentle introduction to automated reasoning Meet Amazon Science's newest research area. Amazon Science: The conference name has the word "logic" in it. Does FLoC deal with other aspects of logic, or is logic coextensive with automated reasoning now? Byron Cook: It's about the intersection of logic and computer science. Automated reasoning is one dimension of that intersection. Daniel Kroning: Traditionally, FLoC is split into two halves, with the first half more theoretical and the second half more applied. Cook: One of the things about automated reasoning is you're on the bleeding edge of what is even computable. We're often working on intractable or undecidable problems. So people automating reasoning are really paying attention to both the applied and the theoretical. AS: I know Marijn is concentrating on SAT solvers, and SAT is an intractable problem, right? It's NP-complete? Marijn Heule: Yes, but you can also use these techniques to solve problems that go beyond NP. For example, solvers for SAT modulo theories, called SMT. I even have a project with one student trying to solve the famous Collatz conjecture with these tools. Collatz-27.png The Collatz conjecture posits that any integer will be transformed into the integer 1 through iterative application of two operations: n /2 and 3n+1. This figure shows a "Collatz cascade" of possible transitions from 27 to 1 using a set of seven symbols, which can be interpreted as simple calculations, and 11 rules for transforming those symbols into symbols consistent with the Collatz operations. At top right are the symbol rewrite rules; at bottom left is a blowup of part of the cascade, illustrating sequences of rewrites that yield the number 425 and its transformation through Collatz operations. Kroning: SAT is now the inexpensive, easy-to-solve workhorse for really hard problems. People still have it in their heads that SAT equals NP hard, therefore difficult to solve or impossible to solve. But for us, it's the lowest entry point. On top of SAT, we build algorithms for solving problems that are way harder. Cook: One of the tricks of the trade is abstraction, where you take a problem that's much, much bigger but represent it with something smaller, where classes of questions you might ask about the smaller problem imply that the answer also holds for the bigger problem. We also have techniques for refining the abstractions on demand when the abstraction is losing too much information to answer the question. Often we can represent these abstractions in tools for SAT. SAT graphs 16x9.png Related content Automated reasoning's scientific frontiers Distributing proof search, reasoning about distributed systems, and automating regulatory compliance are just three fruitful research areas. Marijn's work on the Collatz conjecture is a great example of this. He has done this amazing reduction of Collatz to a series of SAT questions, and he's tantalizingly close to solving it because he's got one decidable problem to go -- and he's the world expert on solving those problems. [Laughs] Heule: Tantalizingly close but also so far away, right? Because this problem might not be solvable even with a million cores. Cook: But it's still decidable. And one of the thresholds is that NP, PSpace, all these things, they're actually decidable. There are questions that are undecidable -- and we work on those, too. When a problem is undecidable, it means that your tool will sometimes fail to find an answer, and that's just fundamental: there are no extra computers you could use ever to solve that. The halting problem is a great example of that. Heule: For these kinds of problems, you're asking the question "Is there a termination argument of this kind of shape?" And if there is one, you have your termination argument. If there is no termination argument of that shape, there could be one of another shape. So if the answer is SAT [satisfiable], then you're happy because you've solved the problem. If the answer is no, you try something else. Cook: It's really, really exciting. In Amazon, we're building these increasingly powerful SAT solvers, using the power of the cloud and distributed systems. So there's no better place for Marijn to be than at Amazon. Embedded verification code.png Related content How to integrate formal proofs into software development ICSE paper presents techniques piloted by Amazon Web Services' Automated Reasoning team. AS: Daniel, could we talk a little bit about your research? Kroning: What I'm looking at right now is reasoning about the cloud infrastructure that performs remote management of EC2 instances -- how to secure that in a way that is provable. You also want to do that in a way that is economical. Cook: One of the things that Daniel's focusing on is agents. We have pieces of software that run on other machines, like EC2 instances, agents for telemetry or for control, and you give them power to take action on your behalf on your machine. But you want to make sure that an adversary doesn't trick those agents into doing bad things. Correct software AS: I know that, commercially, formal methods have been used in hardware design and transportation systems for some time. But it seems that they're really starting to make inroads in software development, too. The storage team is able to write code that otherwise they might not want to deploy because they wouldn't be as confident about it, and they're deploying four times as fast. It was an investment in agility that's really paid off. Byron Cook Cook: The thing we've seen is it's really by need. The storage team, for example, is able to be much more agile and be much more aggressive in the programs that they write because of the formal methods. They're able to write code that otherwise they might not want to deploy because they wouldn't be as confident about it, and they're deploying four times as fast. It was an investment in agility that's really paid off. Kroning: There are actually a good number of stories wherein engineering teams didn't dare to roll out a particular feature or design revision or design variant that offers clear benefits -- like being faster, using less power -- because they just couldn't gain the confidence that it's actually right under all circumstances. Heule: The interesting thing is that you even see this now in tools. Now we have produced proofs from the tools, and people start implementing features that they wouldn't dare have in the past because they were not clear that they were correct. So the solvers get faster and more complex because we now can check the results from the tools and to have confidence in their correctness. ResetOperations_Animation.gif Related content AWS team wins best-paper award for work on automated reasoning SOSP paper describes lightweight formal methods for validating new S3 data storage service. Cook: Yeah, I wanted to double down on that point. There's a distinction in automated reasoning between finding a proof and checking your proof, and the checking is actually relatively easy. It's an accounting thing. Whereas finding the proof is an incredibly creative activity, and the algorithms that find proofs are mind-blowing. But how do you know that the tool that found the proof is correct? Well, you produce an auditable artifact that you can check with the easy tool. SAT in the cloud AS: What are you all most excited about at this year's FLoC? Cook: The SAT conference is at FLoC, and there will be the SAT competition results, and one of the things I'm really excited about is the cloud track. Automated reasoning has really moved into the cloud, and the past couple years running the cloud track has really blown the doors off what's possible. I'm expecting that that will be true again this year. SAT results.png The results of the top-performing cloud-based, parallel, and sequential SAT solvers in this year's SAT competition, whose results were presented at FLoC. The curves show the number of problems (y-axis) in the SAT competition's anniversary problem set -- which aggregates all 5,355 problems presented in the competition's 20-year history -- that a given solver could solve in the allotted time (x-axis). Heule: This is the first year that Amazon is running both the parallel track and the cloud track, and the cloud track was only possible because of Amazon. Before that, there was no way we had the resources to run a cloud track. In the cloud track, every solver-benchmark combination is run on 1,600 cores. And this year is extra special because it's 20 years of SAT, and we have a single anniversary track and all the competitions that were run in the past are in there. That is 5,355 problems, and all the solvers are running on this. Cook: Wow. Heule: I'm also excited to see the results. We have seen in the last year and the year before that the cloud solver can, say, solve in 100 seconds as much as the sequential solvers can do in 5,000 seconds. The user doesn't have to wait for four hours but just for four minutes Cook: And that raises all boats because, as we mentioned earlier, everything is reduced to SAT. If the SAT solvers go from one hour to one minute, that's really game changing. That means a whole other set of things you can do. What has been clear for a while but continues to be true is there's some sort of Moore's-law thing happening with SAT. You fix the same hardware, the same benchmarks, and then run all the tools from the past 20 years, and you see every year they're getting dramatically better. What's also really amazing is that in many ways the tools are getting simpler. LH: Are the simplicity and efficiency two sides of the same coin? Understanding the problems better helps you find a simpler solution, which is more efficient? Cook: Yeah, but it's also the point that Marijn made that because the tools produce auditable proofs that you can check independently, you can do aggressive things that we were scared to do before. Often, aggressive is much simpler. Differential-Cost-Analysis_16x9.gif Related content Calculating the differential cost of code changes Automated-reasoning method enables the calculation of tight bounds on the use of resources -- such as computation or memory -- that results from code changes. Heule: It's also the case that we now understand there are different kinds of problems, and they need different kinds of heuristics. Solvers are combining different heuristics and have phases: "Let's first try this. Let's also try that." And the code involved in changing the heuristics is very small. It's just changing a couple of parameters. But if you notice, okay, this set of heuristics works well for this problem, then you kind of focus more on that. Cook: One of the things a SAT solver does is make decisions fast. It just makes a bunch of choices, and those choices won't work out, and then it spends some time to learn lessons why. And then it has a very efficient internal database for managing what has been learned, what not to do in the future. And that prunes the search space a lot. One of the really exciting things that's happening in the cloud is that you have, say, 1,000 SAT solvers all running on the same problem, and they're learning different things and can share that information amongst them. So by adding 5,000 more solvers, if you can make the communication and the lookup efficient between them, you're really off to the races. The other thing that's quite neat about that is the point that Marijn is making: it's becoming increasingly clear that there are these fundamental building blocks, and for different kinds of problems, you would want to use one kind of Lego brick versus a different kind of Lego brick. And the cloud allows you to run them all but then to share the information between them. Iterated SAT solver.png In "Migrating solver state", Heule and his colleagues show that passing modified versions of a problem between different solvers can accelerate convergence on a solution. Heule: We have an Amazon paper at FLoC with some very cool ideas. If you run things in the cloud, you sometimes have a limited time window where you have to solve them, and otherwise it stops. You started with a certain problem, the solver did some modifications, and now we have a different problem. Initially we just tested, Okay, can we stop the solver and then store the modified problem somewhere and continue later, in case we need more time than we allocated initially? And then we can continue solving it. But the interesting thing is that if you give the modified problem to another solver, and you give it, say, a couple of minutes, and then it stores the modified problem, and you give it to another solver, it actually really speeds things up. It turns out to solve the most instances from everything that we tried. AS: Do you do that in a principled way, or do you just pick a new solver randomly? BugBearScreenshot.large.png Related content How automated reasoning improves the Prime Video experience In a pilot study, an automated code checker found about 100 possible errors, 80% of which turned out to require correction. Heule: The thing that turned out to work really well is to take two top-tier solvers and just Ping-Pong the problem among them. This functionality of storing and continuing search requires some work, so that implementing it in, say, a dozen solvers would require quite some work. But it would be a very interesting experiment. AS: I'm sure our readers would love to know the result of that experiment! Well, thank you all very much for your time. Does anyone have any last thoughts? Cook: I think I speak for the thousands of others who are attending FLoC: we are ready to having our minds blown, just as we did in 2018. Many of the tools and theories presented by our scientific colleagues at this year's FLoC will challenge our current assumptions or spark that next big insight in our brains. We will also get to catch up with old friends that we've known for around 20 years and meet new ones. I'm particularly excited to meet the new generation of scientists who have entered the field, to see the world afresh through their eyes. This is such an amazing time to be in the field of automated reasoning. Research areas * Automated reasoning Tags * Formal methods * Distributed computing * Academics at Amazon Conference * FLoC 2022 Related publications * Migrating solver state About the Author Larry Hardesty Larry Hardesty is the editor of the Amazon Science blog. Previously, he was a senior editor at MIT Technology Review and the computer science writer at the MIT News Office. Related content * SAT graphs 16x9.png Automated reasoning's scientific frontiers Byron Cook February 10, 2022 Distributing proof search, reasoning about distributed systems, and automating regulatory compliance are just three fruitful research areas. Automated reasoning * Policy-code.gif A gentle introduction to automated reasoning Byron Cook December 01, 2021 Meet Amazon Science's newest research area. Automated reasoning * Ranjit_Jhala_ACM Fellow.jpg Amazon Scholar Ranjit Jhala named ACM Fellow Staff writer February 14, 2022 Jhala received the ACM honor for lifetime contributions to software verification, developing innovative tools to help computer programmers test their code. Automated reasoning Work with us See more jobs See more jobs Applied Scientist- AWS AI US, CA, Santa Clara Job summaryAmazon is looking for a passionate, talented, and inventive Applied Scientist with a strong machine learning background to help build industry-leading language technology.Our mission is to provide a delightful experience to Amazon's customers by pushing the envelope in Natural Language Processing (NLP), Natural Language Understanding (NLU), Dialog management, conversational AI and Machine Learning (ML).As part of our AI team in Amazon AWS, you will work alongside internationally recognized experts to develop novel algorithms and modeling techniques to advance the state-of-the-art in human language technology. Your work will directly impact millions of our customers in the form of products and services, as well as contributing to the wider research community. You will gain hands on experience with Amazon's heterogeneous text and structured data sources, and large-scale computing resources to accelerate advances in language understanding.We are hiring primarily in Conversational AI / Dialog System Development areas: NLP, NLU, Dialog Management, NLG.This role can be based in NYC, Seattle or Palo Alto.Inclusive Team CultureHere at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences.Work/ Life BalanceOur team puts a high value on work-life balance. It isn't about how many hours you spend at home or at work; it's about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we're building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future. Applied Scientist- AWS AI US, NY, New York Job summaryAmazon is looking for a passionate, talented, and inventive Applied Scientist with a strong machine learning background to help build industry-leading language technology.Our mission is to provide a delightful experience to Amazon's customers by pushing the envelope in Natural Language Processing (NLP), Natural Language Understanding (NLU), Dialog management, conversational AI and Machine Learning (ML).As part of our AI team in Amazon AWS, you will work alongside internationally recognized experts to develop novel algorithms and modeling techniques to advance the state-of-the-art in human language technology. Your work will directly impact millions of our customers in the form of products and services, as well as contributing to the wider research community. You will gain hands on experience with Amazon's heterogeneous text and structured data sources, and large-scale computing resources to accelerate advances in language understanding.We are hiring primarily in Conversational AI / Dialog System Development areas: NLP, NLU, Dialog Management, NLG.This role can be based in NYC, Seattle or Palo Alto.Inclusive Team CultureHere at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences.Work/ Life BalanceOur team puts a high value on work-life balance. It isn't about how many hours you spend at home or at work; it's about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we're building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future. Research Scientist II, Funnel Science and Analytics US, WA, Seattle Job summaryWorkforce Staffing (WFS) brings together the workforce powering Amazon's ability to delight customers: the Amazon Associate. With over 1M hires, WFS supports sourcing, hiring, and developing the best talent to work in our fulfillment centers, sortation centers, delivery stations, shopping sites, Prime Air locations, and more.WFS' Funnel Science and Analytics team is looking for a Research Scientist. This individual will be responsible for conducting experiments and evaluating the impact of interventions when conducting experiments is not feasible. The perfect candidate will have the applied experience and the theoretical knowledge of policy evaluation and conducting field studies.Key job responsibilitiesAs a Research Scientist (RS), you will do causal inference, design studies and experiments, leverage data science workflows, build predictive models, conduct simulations, create visualizations, and influence science and analytics practice across the organization.Provide insights by analyzing historical data from databases (Redshift, SQL Server, Oracle DW, and Salesforce).Identify useful research avenues for increasing candidate conversion, test, and create well written documents to communicate to technical and non-technical audiences.About the teamFunnel Science and Analytics team finds ways to maximize the conversion and early retention of every candidate who wants to be an Amazon Associate. By focusing on our candidates, we improve candidate and business outcomes, and Amazon takes a step closer to being Earth's Best Employer. Applied Scientist, AI Research & Education US, CA, Santa Clara Job summaryAWS AI/ML is looking for world class scientists and engineers to join its AI Research and Education group working on building automated ML solutions for planetary-scale sustainability and geospatial applications. Our team's mission is to develop ready-to-use and automated solutions that solve important sustainability and geospatial problems. We live in a time wherein geospatial data, such as climate, agricultural crop yield, weather, landcover, etc., has become ubiquitous. Cloud computing has made it easy to gather and process the data that describes the earth system and are generated by satellites, mobile devices, and IoT devices. Our vision is to bring the best ML/AI algorithms to solve practical environmental and sustainability-related R&D problems at scale. Building these solutions require a solid foundation in machine learning infrastructure and deep learning technologies. The team specializes in developing popular open source software libraries like AutoGluon, GluonCV, GluonNLP, DGL, Apache/MXNet (incubating). Our strategy is to bring the best of ML based automation to the geospatial and sustainability area.We are seeking an experienced Applied Scientist for the team. This is a role that combines science knowledge (around machine learning, computer vision, earth science), technical strength, and product focus. It will be your job to develop ML system and solutions and work closely with the engineering team to ship them to our customers. You will interact closely with our customers and with the academic and research communities. You will be at the heart of a growing and exciting focus area for AWS and work with other acclaimed engineers and world famous scientists. You are also expected to work closely with other applied scientists and demonstrate Amazon Leadership Principles (https://www.amazon.jobs/en/ principles). Strong technical skills and experience with machine learning and computer vision are required. Experience working with earth science, mapping, and geospatial data is a plus. Our customers are extremely technical and the solutions we build for them are strongly coupled to technical feasibility.About the teamInclusive Team CultureAt AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon's culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Work/Life BalanceOur team puts a high value on work-life balance. It isn't about how many hours you spend at home or at work; it's about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we're building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded scientist and enable them to take on more complex tasks in the future.Interested in this role? Reach out to the recruiting team with questions or apply directly via amazon.jobs. Applied Scientist, AI Research & Education US, CA, Santa Clara Job summaryAWS AI/ML is looking for world class scientists and engineers to join its AI Research and Education group working on building automated ML solutions for planetary-scale sustainability and geospatial applications. Our team's mission is to develop ready-to-use and automated solutions that solve important sustainability and geospatial problems. We live in a time wherein geospatial data, such as climate, agricultural crop yield, weather, landcover, etc., has become ubiquitous. Cloud computing has made it easy to gather and process the data that describes the earth system and are generated by satellites, mobile devices, and IoT devices. Our vision is to bring the best ML/AI algorithms to solve practical environmental and sustainability-related R&D problems at scale. Building these solutions require a solid foundation in machine learning infrastructure and deep learning technologies. The team specializes in developing popular open source software libraries like AutoGluon, GluonCV, GluonNLP, DGL, Apache/MXNet (incubating). Our strategy is to bring the best of ML based automation to the geospatial and sustainability area.We are seeking an experienced Applied Scientist for the team. This is a role that combines science knowledge (around machine learning, computer vision, earth science), technical strength, and product focus. It will be your job to develop ML system and solutions and work closely with the engineering team to ship them to our customers. You will interact closely with our customers and with the academic and research communities. You will be at the heart of a growing and exciting focus area for AWS and work with other acclaimed engineers and world famous scientists. You are also expected to work closely with other applied scientists and demonstrate Amazon Leadership Principles (https://www.amazon.jobs/en/ principles). Strong technical skills and experience with machine learning and computer vision are required. Experience working with earth science, mapping, and geospatial data is a plus. Our customers are extremely technical and the solutions we build for them are strongly coupled to technical feasibility.About the teamInclusive Team CultureAt AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon's culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Work/Life BalanceOur team puts a high value on work-life balance. It isn't about how many hours you spend at home or at work; it's about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we're building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded scientist and enable them to take on more complex tasks in the future.Interested in this role? Reach out to the recruiting team with questions or apply directly via amazon.jobs. Senior Applied Scientist - Machine Learning, Personalization, Recommendations, Machine Learning, Causal Inference, Personalization US, WA, Seattle Job summaryHow can we create a rich, data-driven shopping experience on Amazon? How do we build data models that helps us innovate different ways to enhance customer experience? How do we combine the world's greatest online shopping dataset with Amazon's computing power to create models that deeply understand our customers? Recommendations at Amazon is a way to help customers discover products. Our team's stated mission is to "grow each customer's relationship with Amazon by leveraging our deep understanding of them to provide relevant and timely product, program, and content recommendations". We strive to better understand how customers shop on Amazon (and elsewhere) and build recommendations models to streamline customers' shopping experience by showing the right products at the right time. Understanding the complexities of customers' shopping needs and helping them explore the depth and breadth of Amazon's catalog is a challenge we take on every day. Using Amazon's large-scale computing resources you will ask research questions about customer behavior, build models to generate recommendations, and run these models directly on the retail website. You will participate in the Amazon ML community and mentor Applied Scientists and software development engineers with a strong interest in and knowledge of ML. Your work will directly benefit customers and the retail business and you will measure the impact using scientific tools. We are looking for passionate, hard-working, and talented Applied scientist who have experience building mission critical, high volume applications that customers love. You will have an enormous opportunity to make a large impact on the design, architecture, and implementation of cutting edge products used every day, by people you know.Key job responsibilitiesScaling state of the art techniques to Amazon-scaleWorking independently and collaborating with SDEs to deploy models to productionDeveloping long-term roadmaps for the team's scientific agendaDesigning experiments to measure business impact of the team's effortsMentoring scientists in the departmentContributing back to the machine learning science community Applied Scientist US, NY, New York City Job summaryAmazon Web Services is looking for world class scientists to join the Security Analytics and AI Research team within AWS Security Services. This group is entrusted with researching and developing core data mining and machine learning algorithms for various AWS security services like GuardDuty (https://aws.amazon.com/ guardduty/) and Macie (https://aws.amazon.com/macie/). In this group, you will invent and implement innovative solutions for never-before-solved problems. If you have passion for security and experience with large scale machine learning problems, this will be an exciting opportunity.The AWS Security Services team builds technologies that help customers strengthen their security posture and better meet security requirements in the AWS Cloud. The team interacts with security researchers to codify our own learnings and best practices and make them available for customers. We are building massively scalable and globally distributed security systems to power next generation services.Inclusive Team Culture Here at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon's culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Work/Life Balance Our team puts a high value on work-life balance. It isn't about how many hours you spend at home or at work; it's about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives. Mentorship & Career Growth Our team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we're building an environment that celebrates knowledge sharing and mentorship. We care about your career growth and strive to assign projects based on what will help each team member develop and enable them to take on more complex tasks in the future.A day in the lifeAbout the hiring groupJob responsibilities* Rapidly design, prototype and test many possible hypotheses in a high-ambiguity environment, making use of both quantitative and business judgment.* Collaborate with software engineering teams to integrate successful experiments into large scale, highly complex production services.* Report results in a scientifically rigorous way.* Interact with security engineers, product managers and related domain experts to dive deep into the types of challenges that we need innovative solutions for. Economist II - AMZ6003203 US, CA, San Francisco Job summaryMULTIPLE POSITIONS AVAILABLECompany: AMAZON.COM SERVICES LLCPosition Title: Economist IILocation: San Francisco, CaliforniaPosition Responsibilities:Work with fellow economists, scientists and/or senior management on key business problems faced in retail, international retail, third party merchants, search, and/or operations. Apply the frontier of economic thinking to experiment design, forecasting, program evaluation and other areas. Build econometric models using data systems. Apply economic theory to solve business problems. Own the development of economic models and manage the data analysis, modeling and experimentation necessary to estimate and validate the models, in collaboration with scientists and engineers. Develop new techniques to process large data sets, address quantitative problems, and contribute to design of automated systems. Apply tools from applied micro-econometrics (e.g. experimental design, difference-in-difference, regression discontinuity) and forecasting (essential time series models). Leverage big data tools for data extraction. Work closely with business partners to communicate the intuition, implication and detail of economic analyses/modeling and incorporate feedback. Write up and present analysis for distribution to various levels of management at Amazon.Amazon.com is an Equal Opportunity-Affirmative Action Employer - Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation.#0000 Sr. Applied Scientist - Machine Learning US, CA, Palo Alto Job summaryAmazon Advertising is one of Amazon's fastest growing and most profitable businesses, responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our products and solutions are strategically important to enable our Retail and Marketplace businesses to drive long-term growth. We deliver billions of ad impressions and millions of clicks and break fresh ground in product and technical innovations every day!The Advertising Forecasting Science team comprises top scientists and engineers based in Palo Alto, California. The team builds forecasting models for advertising campaigns and financial planning, with revenue exceeding tens of billions of dollars. The forecasting science team makes auction prediction and handles bid optimization for billions of daily requests using innovative machine learning algorithms to optimize performance, which generates billions of annual revenue!As an Applied Scientist on this team, you will: Develop scalable and effective machine Learning models with automated training, validation, monitoring and reporting.Work with talented scientists and engineers to solve problems in the domains of forecasting, auction theory, bid optimization, and user clustering.Conduct deep data analyses on massive ad user and contextual data sets.Invent ways to overcome technical limitations and enable new forms of analyses to drive key technical and business decisions.Stay familiar with the field and apply state-of-the-art machine learning techniques to our domain problems, around forecasting, bidding, allocation, and optimization.Produce peer-reviewed scientific paper in top journals and conferences.Present results, reports, and data insights to both technical and business leadership.Why you will love this opportunity: Amazon is investing heavily in building a world-class advertising business. This team defines and delivers a collection of advertising products that drive discovery and sales. Our solutions generate billions in revenue and drive long-term growth for Amazon's Retail and Marketplace businesses. We deliver billions of ad impressions, millions of clicks daily, and break fresh ground to create world-class products. We are a highly motivated, collaborative, and fun-loving team with an entrepreneurial spirit - with a broad mandate to experiment and innovate.Impact and Career Growth: You will invent new experiences and influence customer-facing shopping experiences to help suppliers grow their retail business and the auction dynamics that leverage native advertising; this is your opportunity to work within the fastest-growing businesses across all of Amazon! Define a long-term science vision for our advertising business, driven from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding.Team video https://youtu.be/zD_6Lzw8raE Manager, Applied Science CV/ML, Robotics AI US, WA, Seattle Job summaryDo you want to have a worldwide impact in Robotics? The Robotics AI team at Amazon builds high-performance, real-time robotic systems that can perceive, learn, and act intelligently alongside humans--at Amazon scale. We invent and scale AI systems for robotics in fulfillment. Our mission is to enable robots to interact safely, efficiently, and fluently with the clutter and uncertainty of real-world fulfillment centers. We hire and develop subject matter experts in robotics with a focus on computer vision, deep learning, intelligent control, semi-supervised and unsupervised learning. We are seeking hands-on, Applied Science Manager to own the development of Perception and Task planning algorithms to advance robotics in our fulfillment network along with leading teams. You will be deep in algorithms and code. A successful candidate would be an experienced people manager with good leadership skills combined with excellent technical depth in Computer vision/ Deep Learning/ Perception systems / Task planning, great communication skills, and a drive to achieve results in a collaborative team environment. In this role you will provide people management and also apply the latest trends in research to solve real-world problems. You will be an integral part of the core robotics team and work with others to implement robotics systems above and beyond the current state-of-the-art in the field.You should enjoy the process of solving real-world problems that, quite frankly, haven't been solved at scale anywhere before. Along the way, we guarantee you'll get opportunities to be a fearless disruptor, prolific innovator, and a reputed problem solver--someone who truly enables AI and robotics to significantly impact the lives of millions of consumers.Key job responsibilities* As a manager, you will be responsible for delivering and maintaining critical robotic capabilities in the fulfillment network.* You will drive your team to research, design, implement and evaluate complex perception and decision making algorithms integrating across multiple disciplines* You will prioritize being a great people manager: motivating, rewarding, and coaching your diverse team is the most important part of this role. You will recruit and retain top talent and excel in day-to-day people and performance management tasks.* You will keep your technical skills current to contribute to architecture and design discussions. * You will regularly take part in deep-dive exercises and drive technical post-mortem discussions to identify the root cause of complex issues.* Set a vision for your team and create product roadmaps. Help your team sort out technical and product requirements and priorities. Use project management skills to deliver product roadmap items and other cross-team projects. [amazon-science-logo-whi] * About * Research areas * Blog * News and features * Publications * Conferences * Collaborations * Careers * Alexa Prize * Academics * Research Awards * Amazon Developer * Amazon Web Services * About Amazon * Newsletter * RSS View from space of a connected network around planet Earth representing the Internet of Things. Have feedback for Amazon Science? We want to hear from you. Take survey Amazon.com | Conditions of Use | Privacy | (c) 1996-2022 Amazon.com, Inc. or its affiliates Follow Us * twitter * instagram * youtube * facebook * linkedin