https://testing.googleblog.com/2014/04/the-real-test-driven-development.html

*
[googlelogo]

Testing Blog

The *Real* Test Driven Development

Tuesday, April 01, 2014

Update: APRIL FOOLS!


by Kaue Silveira

Here at Google, we invest heavily in development productivity
research. In fact, our TDD research group now occupies nearly an
entire building of the Googleplex. The group has been working hard to
minimize the development cycle time, and we'd like to share some of
the amazing progress they've made.

The Concept

In the ways of old, it used to be that people wrote tests for their
existing code. This was changed by TDD (Test-driven Development),
where one would write the test first and then write the code to
satisfy it. The TDD research group didn't think this was enough and
wanted to elevate the humble test to the next level. We are pleased
to announce the Real TDD, our latest innovation in the Program
Synthesis field, where you write only the tests and have the computer
write the code for you!

The following graph shows how the number of tests created by a small
feature team grew since they started using this tool towards the end
of 2013. Over the last 2 quarters, more than 89% of this team's
production code was written by the tool!

[image]
See it in action:

Test written by a Software Engineer:


class LinkGeneratorTest(googletest.TestCase):

  def setUp(self):
    self.generator = link_generator.LinkGenerator()

  def testGetLinkFromIDs(self):
    expected = ('https://frontend.google.com/advancedSearchResults?'
                's.op=ALL&s.r0.field=ID&s.r0.val=1288585+1310696+1346270+')
    actual = self.generator.GetLinkFromIDs(set((1346270, 1310696, 1288585)))
    self.assertEqual(expected, actual)


Code created by our tool:


import urllib

class LinkGenerator(object):

  _URL = (
      'https://frontend.google.com/advancedSearchResults?'
      's.op=ALL&s.r0.field=ID&s.r0.val=')

  def GetLinkFromIDs(self, ids):
    result = []
    for id in sorted(ids):
      result.append('%s ' % id)
    return self._URL + urllib.quote_plus(''.join(result))


Note that the tool is smart enough to not generate the obvious
implementation of returning a constant string, but instead it
correctly abstracts and generalizes the relation between inputs and
outputs. It becomes smarter at every use and it's behaving more and
more like a human programmer every day. We once saw a comment in the
generated code that said "I need some coffee".

How does it work?

We've trained the Google Brain with billions of lines of open-source
software to learn about coding patterns and how product code
correlates with test code. Its accuracy is further improved by using
Type Inference to infer types from code and the Girard-Reynolds
Isomorphism to infer code from types.

The tool runs every time your unit test is saved, and it uses the
learned model to guide a backtracking search for a code snippet that
satisfies all assertions in the test. It provides sub-second
responses for 99.5% of the cases (as shown in the following graph),
thanks to millions of pre-computed assertion-snippet pairs stored in
Spanner for global low-latency access.

[image]


How can I use it?

We will offer a free (rate-limited) service that everyone can use,
once we have sorted out the legal issues regarding the possibility of
mixing code snippets originating from open-source projects with
different licenses (e.g., GPL-licensed tests will simply refuse to
pass BSD-licensed code snippets). If you would like to try our alpha
release before the public launch, leave us a comment!

Share on Twitter Share on Facebook
Google
Labels: April Fools , Kaue Silveira

33 comments :

 1. [blogger_lo]
    Christian SeitzerApril 1, 2014 at 2:50:00 AM PDT

    This is my favorite april fool's day prank up to now. :)

    As a next step i suggest you find a way to automate the test
    writing.
    Oh, and please sign me up for your alpha release ;)

    ReplyDelete
    Replies
        Reply
 2. [blogger_lo]
    UnknownApril 1, 2014 at 3:16:00 AM PDT

    This is 1st April joke or real ?)

    ReplyDelete
    Replies
        Reply
 3. [blogger_lo]
    Alain ChautardApril 1, 2014 at 5:05:00 AM PDT

    April 1st 2014 :-)

    ReplyDelete
    Replies
        Reply
 4. [blogger_lo]
    Franklin ChenApril 1, 2014 at 7:01:00 AM PDT

    For the record, the code inferencer demo program mentioned, Djinn
    is available at http://hackage.haskell.org/package/djinn and can
    be installed as the Cabal package "djinn" if you have the Haskell
    Platform installed. Also, you can do this sort of code
    inferencing in straight Haskell: see Oleg's demo code here: http:
    //okmij.org/ftp/Haskell/types.html#de-typechecker

    Parametricity is great. In all seriousness, and not just on this
    first day of the month!

    ReplyDelete
    Replies
        Reply
 5. [blogger_lo]
    PJayTycyApril 1, 2014 at 7:49:00 AM PDT

    Will I be able to add security loopholes in Google production
    code by submitting such code + some innocent testcases to some
    unknown open source project?

    ReplyDelete
    Replies
        Reply
 6. [blogger_lo]
    UnknownApril 1, 2014 at 10:16:00 AM PDT

    Does the team now only write tests for "real TDD"?

    ReplyDelete
    Replies
        Reply
 7. [blogger_lo]
    UnknownApril 1, 2014 at 2:00:00 PM PDT

    I am interested to alpha test this public Program synthesis tool

    ReplyDelete
    Replies
        Reply
 8. [blogger_lo]
    UnknownApril 1, 2014 at 2:26:00 PM PDT

    Innovative stuff!

    ReplyDelete
    Replies
        Reply
 9. [blogger_lo]
    ChrisApril 1, 2014 at 4:06:00 PM PDT

    Best one today!

    ReplyDelete
    Replies
        Reply
10. [blogger_lo]
    FelipeApril 1, 2014 at 4:07:00 PM PDT

    Very interesting, I definitely want to try the alpha release!

    ReplyDelete
    Replies
        Reply
11. [blogger_lo]
    UnknownApril 1, 2014 at 5:14:00 PM PDT

    If this is for real, I'd love to try this else it's a good prank,
    google!

    ReplyDelete
    Replies
        Reply
12. [blogger_lo]
    ParvaApril 1, 2014 at 7:36:00 PM PDT

    This is very clever prank! Why don't we train Google Brain to
    write code that requires less testing... at least avoiding
    mistakes that humans make :-) I'll probably pass on alpha
    opportunity, but do count me in for beta!

    ReplyDelete
    Replies
        Reply
13. [blogger_lo]
    UnknownApril 1, 2014 at 8:26:00 PM PDT

    It's *REAL* real?

    ReplyDelete
    Replies
        Reply
14. [blogger_lo]
    UnknownApril 1, 2014 at 10:55:00 PM PDT

    Is it prank or real?

    ReplyDelete
    Replies
        Reply
15. [blogger_lo]
    UnknownApril 1, 2014 at 11:16:00 PM PDT

    where one would write the test first and then write the code to
    satisfy it.

    I don't think it would be possible to write the test case and the
    write the code to satisfy it. If it is possible , it will be the
    revolution in software era.

    ReplyDelete
    Replies
        Reply
16. [blogger_lo]
    UnknownApril 2, 2014 at 5:40:00 AM PDT

    Kaue, could you spoil Mew's location, please? Kidding!!!
    BTW, I hope this isn't another prank.. got really excited about
    it! Great job, anyways!
    Cheers!

    ReplyDelete
    Replies
        Reply
17. [blogger_lo]
    UnknownApril 2, 2014 at 7:40:00 AM PDT

    Awesome. Is this going to be available for Java as well? The
    example snippet seems to be in Python.

    ReplyDelete
    Replies
        Reply
18. [blogger_lo]
    UnknownApril 2, 2014 at 3:14:00 PM PDT

    I am such a sucker.

    ReplyDelete
    Replies
        Reply
19. [Foto0024_r]
    Matias M.April 2, 2014 at 3:29:00 PM PDT

    =) hahahaha very funny

    ReplyDelete
    Replies
        Reply
20. [blank]
    AnonymousApril 2, 2014 at 4:32:00 PM PDT

    Fiction imitates real life: I just published a PLDI'14 paper on
    actually (no joke) generating code for TDD: https://
    homes.cs.washington.edu/~perelman/publications/pldi14-tds.pdf
    Unfortunately, I have not yet achieved 100x programmer
    productivity improvements using the real thing.

    ReplyDelete
    Replies
        Reply
21. [MyPic]
    Infant GeraldApril 2, 2014 at 5:42:00 PM PDT

    I dont think this is impossible. The possibility of auto code
    generation that matches your tests is smart way for going about
    writing all possible used cases. How ever I am wondering if there
    is a possibility to even write all test cases given a few cases
    are written.
    So humans write a few testcases, Code writes more edge test cases
    be following QA guidelines. These then drive production code. I
    think this is effective utilization of human knowledge

    ReplyDelete
    Replies
        Reply
22. [Photo]
    LorenzoApril 3, 2014 at 3:30:00 AM PDT

    I'd like to please consider me for the alpha release

    ReplyDelete
    Replies
        Reply
23. [blogger_lo]
    UnknownApril 4, 2014 at 9:22:00 AM PDT

    This is the only April Fool's prank that got me this year...

    ReplyDelete
    Replies
        Reply
24. [blogger_lo]
    jkpdApril 5, 2014 at 5:04:00 PM PDT

    I don't bite it! I want to see that!
    So my days are numbered as a developer...Never too late to catch
    my guitar again and hit the road :-)

    ReplyDelete
    Replies
        Reply
25. [blogger_lo]
    MuditApril 7, 2014 at 1:09:00 AM PDT

    Please Consider me for Alpha Release.

    ReplyDelete
    Replies
        Reply
26. [Rob]
    Rob BaillieApril 11, 2014 at 2:25:00 AM PDT

    I'll start with of course this was an April fool's joke. Wow.

    But for those getting excited at the prospect...

    Of course, writing tests takes about 100th of the time of
    designing application structure and the writing code, doesn't it?
    The productivity gains would be nothing like as high as the
    suggestion.

    My guess would be that best one could hope for in a tool such as
    this (when it finally does appear) would be around doubling the
    output.

    Having something write code for you may be fantasy today, but
    reality soon; the productivity gains are just PURE fantasy.

    ReplyDelete
    Replies
        Reply
27. [blogger_lo]
    UnknownApril 17, 2014 at 11:14:00 AM PDT

    real intresting stuff, would like to try it

    ReplyDelete
    Replies
        Reply
28. [blogger_lo]
    La IndieApril 21, 2014 at 9:28:00 AM PDT

    I want to try it!

    ReplyDelete
    Replies
        Reply
29. [blogger_lo]
    UnknownApril 24, 2014 at 8:59:00 PM PDT

    I want to real TDD

    ReplyDelete
    Replies
        Reply
30. [blogger_lo]
    Simple CodingMay 7, 2014 at 2:07:00 AM PDT

    if there were one, I wanna try it.

    ReplyDelete
    Replies
        Reply
31. [blogger_lo]
    StunamiMay 21, 2014 at 8:18:00 PM PDT

    This sounds great! I'd like to try it.

    ReplyDelete
    Replies
        Reply
32. [blogger_lo]
    Shaked Klein OrbachMay 24, 2014 at 4:55:00 AM PDT

    Sounds like an interesting idea, would be happy to try it!

    Thank you

    ReplyDelete
    Replies
        Reply
33. [blogger_lo]
    UnknownDecember 4, 2014 at 10:32:00 AM PST

    I got to "where you write only the tests and have the computer
    write the code for you!" before checking the publication date.

    ReplyDelete
    Replies
        Reply

Add comment
Load more...

The comments you read and contribute here belong only to the person
who posted them. We reserve the right to remove off-topic comments.

 

  
[                    ]
[ncccFICB87]

Labels



  * TotT 104
  * GTAC 61
  * James Whittaker 42
  * Misko Hevery 32
  * Code Health 31
  * Anthony Vallone 27
  * Patrick Copeland 23
  * Jobs 18
  * Andrew Trenk 13
  * C++ 11
  * Patrik Hoglund 8
  * JavaScript 7
  * Allen Hutchison 6
  * George Pirocanac 6
  * Zhanyong Wan 6
  * Harry Robinson 5
  * Java 5
  * Julian Harty 5
  * Adam Bender 4
  * Alberto Savoia 4
  * Ben Yu 4
  * Erik Kuefler 4
  * Philip Zembrod 4
  * Shyam Seshadri 4
  * Chrome 3
  * Dillon Bly 3
  * John Thomas 3
  * Lesley Katzen 3
  * Marc Kaplan 3
  * Markus Clermont 3
  * Max Kanat-Alexander 3
  * Sonal Shah 3
  * APIs 2
  * Abhishek Arya 2
  * Alan Myrvold 2
  * Alek Icev 2
  * Android 2
  * April Fools 2
  * Chaitali Narla 2
  * Chris Lewis 2
  * Chrome OS 2
  * Diego Salas 2
  * Dori Reuveni 2
  * Jason Arbon 2
  * Jochen Wuttke 2
  * Kostya Serebryany 2
  * Marc Eaddy 2
  * Marko Ivankovic 2
  * Mobile 2
  * Oliver Chang 2
  * Simon Stewart 2
  * Stefan Kennedy 2
  * Test Flakiness 2
  * Titus Winters 2
  * Tony Voellm 2
  * WebRTC 2
  * Yiming Sun 2
  * Yvette Nameth 2
  * Zuri Kemp 2
  * Aaron Jacobs 1
  * Adam Porter 1
  * Adam Raider 1
  * Adel Saoud 1
  * Alan Faulkner 1
  * Alex Eagle 1
  * Amy Fu 1
  * Anantha Keesara 1
  * Antoine Picard 1
  * App Engine 1
  * Ari Shamash 1
  * Arif Sukoco 1
  * Benjamin Pick 1
  * Bob Nystrom 1
  * Bruce Leban 1
  * Carlos Arguelles 1
  * Carlos Israel Ortiz Garcia 1
  * Cathal Weakliam 1
  * Christopher Semturs 1
  * Clay Murphy 1
  * Dagang Wei 1
  * Dan Maksimovich 1
  * Dan Shi 1
  * Dan Willemsen 1
  * Dave Chen 1
  * Dave Gladfelter 1
  * David Bendory 1
  * David Mandelberg 1
  * Derek Snyder 1
  * Diego Cavalcanti 1
  * Dmitry Vyukov 1
  * Eduardo Bravo Ortiz 1
  * Ekaterina Kamenskaya 1
  * Elliott Karpilovsky 1
  * Elliotte Rusty Harold 1
  * Espresso 1
  * Felipe Sodre 1
  * Francois Aube 1
  * Gene Volovich 1
  * Google+ 1
  * Goran Petrovic 1
  * Goranka Bjedov 1
  * Hank Duan 1
  * Havard Rast Blok 1
  * Hongfei Ding 1
  * Jason Elbaum 1
  * Jason Huggins 1
  * Jay Han 1
  * Jeff Hoy 1
  * Jeff Listfield 1
  * Jessica Tomechak 1
  * Jim Reardon 1
  * Joe Allan Muharsky 1
  * Joel Hynoski 1
  * John Micco 1
  * John Penix 1
  * Jonathan Rockway 1
  * Jonathan Velasquez 1
  * Josh Armour 1
  * Julie Ralph 1
  * Kai Kent 1
  * Kanu Tewary 1
  * Karin Lundberg 1
  * Kaue Silveira 1
  * Kevin Bourrillion 1
  * Kevin Graney 1
  * Kirkland 1
  * Kurt Alfred Kluever 1
  * Manjusha Parvathaneni 1
  * Marek Kiszkis 1
  * Marius Latinis 1
  * Mark Ivey 1
  * Mark Manley 1
  * Mark Striebeck 1
  * Matt Lowrie 1
  * Meredith Whittaker 1
  * Michael Bachman 1
  * Michael Klepikov 1
  * Mike Aizatsky 1
  * Mike Wacker 1
  * Mona El Mahdy 1
  * Noel Yap 1
  * Palak Bansal 1
  * Patricia Legaspi 1
  * Per Jacobsson 1
  * Peter Arrenbrecht 1
  * Peter Spragins 1
  * Phil Norman 1
  * Phil Rollet 1
  * Pooja Gupta 1
  * Project Showcase 1
  * Radoslav Vasilev 1
  * Rajat Dewan 1
  * Rajat Jain 1
  * Rich Martin 1
  * Richard Bustamante 1
  * Roshan Sembacuttiaratchy 1
  * Ruslan Khamitov 1
  * Sam Lee 1
  * Sean Jordan 1
  * Sebastian Dorner 1
  * Sharon Zhou 1
  * Shiva Garg 1
  * Siddartha Janga 1
  * Simran Basi 1
  * Stan Chan 1
  * Stephen Ng 1
  * Tejas Shah 1
  * Test Analytics 1
  * Test Engineer 1
  * Tim Lyakhovetskiy 1
  * Tom O'Neill 1
  * Vojta Jina 1
  * automation 1
  * dead code 1
  * iOS 1
  * mutation testing 1



Archive



  * >  2025 (1)
      + >  Jan (1)

  * >  2024 (13)
      + >  Dec (1)
      + >  Oct (1)
      + >  Sep (1)
      + >  Aug (1)
      + >  Jul (1)
      + >  May (3)
      + >  Apr (3)
      + >  Mar (1)
      + >  Feb (1)

  * >  2023 (14)
      + >  Dec (2)
      + >  Nov (2)
      + >  Oct (5)
      + >  Sep (3)
      + >  Aug (1)
      + >  Apr (1)

  * >  2022 (2)
      + >  Feb (2)

  * >  2021 (3)
      + >  Jun (1)
      + >  Apr (1)
      + >  Mar (1)

  * >  2020 (8)
      + >  Dec (2)
      + >  Nov (1)
      + >  Oct (1)
      + >  Aug (2)
      + >  Jul (1)
      + >  May (1)

  * >  2019 (4)
      + >  Dec (1)
      + >  Nov (1)
      + >  Jul (1)
      + >  Jan (1)

  * >  2018 (7)
      + >  Nov (1)
      + >  Sep (1)
      + >  Jul (1)
      + >  Jun (2)
      + >  May (1)
      + >  Feb (1)

  * >  2017 (17)
      + >  Dec (1)
      + >  Nov (1)
      + >  Oct (1)
      + >  Sep (1)
      + >  Aug (1)
      + >  Jul (2)
      + >  Jun (2)
      + >  May (3)
      + >  Apr (2)
      + >  Feb (1)
      + >  Jan (2)

  * >  2016 (15)
      + >  Dec (1)
      + >  Nov (2)
      + >  Oct (1)
      + >  Sep (2)
      + >  Aug (1)
      + >  Jun (2)
      + >  May (3)
      + >  Apr (1)
      + >  Mar (1)
      + >  Feb (1)

  * >  2015 (14)
      + >  Dec (1)
      + >  Nov (1)
      + >  Oct (2)
      + >  Aug (1)
      + >  Jun (1)
      + >  May (2)
      + >  Apr (2)
      + >  Mar (1)
      + >  Feb (1)
      + >  Jan (2)

  * V  2014 (24)
      + >  Dec (2)
      + >  Nov (1)
      + >  Oct (2)
      + >  Sep (2)
      + >  Aug (2)
      + >  Jul (3)
      + >  Jun (3)
      + >  May (2)
      + V  Apr (2)
          o Testing on the Toilet: Test Behaviors, Not Methods
          o The *Real* Test Driven Development
      + >  Mar (2)
      + >  Feb (1)
      + >  Jan (2)

  * >  2013 (16)
      + >  Dec (1)
      + >  Nov (1)
      + >  Oct (1)
      + >  Aug (2)
      + >  Jul (1)
      + >  Jun (2)
      + >  May (2)
      + >  Apr (2)
      + >  Mar (2)
      + >  Jan (2)

  * >  2012 (11)
      + >  Dec (1)
      + >  Nov (2)
      + >  Oct (3)
      + >  Sep (1)
      + >  Aug (4)

  * >  2011 (39)
      + >  Nov (2)
      + >  Oct (5)
      + >  Sep (2)
      + >  Aug (4)
      + >  Jul (2)
      + >  Jun (5)
      + >  May (4)
      + >  Apr (3)
      + >  Mar (4)
      + >  Feb (5)
      + >  Jan (3)

  * >  2010 (37)
      + >  Dec (3)
      + >  Nov (3)
      + >  Oct (4)
      + >  Sep (8)
      + >  Aug (3)
      + >  Jul (3)
      + >  Jun (2)
      + >  May (2)
      + >  Apr (3)
      + >  Mar (3)
      + >  Feb (2)
      + >  Jan (1)

  * >  2009 (54)
      + >  Dec (3)
      + >  Nov (2)
      + >  Oct (3)
      + >  Sep (5)
      + >  Aug (4)
      + >  Jul (15)
      + >  Jun (8)
      + >  May (3)
      + >  Apr (2)
      + >  Feb (5)
      + >  Jan (4)

  * >  2008 (75)
      + >  Dec (6)
      + >  Nov (8)
      + >  Oct (9)
      + >  Sep (8)
      + >  Aug (9)
      + >  Jul (9)
      + >  Jun (6)
      + >  May (6)
      + >  Apr (4)
      + >  Mar (4)
      + >  Feb (4)
      + >  Jan (2)

  * >  2007 (41)
      + >  Oct (6)
      + >  Sep (5)
      + >  Aug (3)
      + >  Jul (2)
      + >  Jun (2)
      + >  May (2)
      + >  Apr (7)
      + >  Mar (5)
      + >  Feb (5)
      + >  Jan (4)

[8LnOxYWFhZ]

Feed

[P2DAYDAaDwWA]

  * Google
  * Privacy
  * Terms