I went to a couple of talks by John Carnell on Leadership and the Lean process. The lean practices sounded kind of interesting but I never really figured out exactly what was going on through out the talks. The one part of the lean process that I picked up was making Value Stream Maps to help figure out where your process was wasteful. I don't know that I would ever do that because I think everybody here already recognized where we are wasteful, now the trick is figuring out how to make parts of the process better where they are wasteful. The part that is really good, and I think I will start using is his estimation stuff. He showed a spreadsheet where tasks were estimated as ether 20,40, or 80 hour tasks, and then there was some percentages for prototyping, design, testing, and some different risk factors that would help to generate a low..assume all risk factors were 0% and a high which was calculated given the risk factor percentages that you entered. He offered the spreadsheet to anyone that wanted a copy so I emailed him to get a copy that I can start playing around with. If I come up with something different after I start playing around with it I will post it here.
The last day I spent all day at Jared Richardson's talks on Agile Process type topics. All the talks and topics kinda blended together. One of the big things he kept going over and over was that nothing works if you don't have CI ( continuous integration ) and test automation. That was one thing that really gave me a good feeling about where we are at today in the company. We use Cruise Control and have all of our projects building there and unit tests running when they exist, so at least we have the foundation built. Tracer Bullets was a really neat idea too. Basically set up an API at each layer ( DB, DAO, computation, display, etc. ) that returns hard coded data. Then you can verify before any implementation happens that your protocols work and the communication is there. As far as myself I took a lot of Integration level testing ideas from these talks. Basically I neglect integration, and if there is one thing I took from all of his talks is that integration tests are just as important as unit tests. I liked his idea of writing a very high level integration test first, then implementing the lower level code, writing unit tests as you write code, with the goal of getting the integration tests to pass by completing the units. In the past when I have written integration tests its always been the other way around...write all the units, then as I build up to the integration level I write the integration tests. Another cool testing idea was DDT, defect driven testing. Maybe this can get people on board with Automated Testing in that if something is broken you write a unit test for it. Then as everybody gets better at it, start moving towards TDD. I will continue to say that we should enforce at least unit tests be written on all code checked in. Otherwise we are just going to continue to generate more and more code that is not covered and keep building up the Technical Debt. I have already, unintentionally, done a Blitzkrieg of sorts. As we wanted developers to start writing more tests ( we encourage it, but rarely does it get done ) we went through most of our projects and added a few unit tests to each so that there was examples and the framework set in most every project. Peer Code Reviews are another thing that I think we should start doing. Today I think that my manager is reviewing my code, but generally speaking this is done when we do a QA release, and at the point it is hard to go back and change things. I liked the idea of forcing a code review before checking anything in, and enforcing that you cannot have the same person do 2 code reviews for you. We are a pretty divided department, one team works on one set of code and the other works on another set of code, we are slowly breaking down the barriers between code and everybody is working more on everything. But I liked this idea that somebody on the other team could review your code too, thereby helping them understand what's going on with the other team in the process.
A topic that came up in a lot of talks, not just Jared's, was time boxing iterations. I really liked this idea of breaking down tasks smaller building lists of what you want to get done. I think a lot of us were already close to doing this using SharePoint and/or Xplanner, but listening to all the different ideas I think I feel better about knowing how to do it. On of the keys that I never really thought about is that every task should take between ½ day and 3 days to complete. Along those lines the idea of a daily meeting that is really a stand up couple of minutes to answer 3 questions: What did you do yesterday? What are you going to do today? What problems do you have? Then if you have a visible list everybody knows whats going on all the time. All of these talks and information about lists, daily meetings, etc. lead me to think that we might be better off working in a more team orientated environment. We all work together very well, but maybe instead of each developer getting assigned to a project we assign a project to a group of 2-4 developers. Everybody can work together to come up with a good design and everybody can grab random tasks for code, which would lead to a bit more diversity. I am terrible when it comes to writing a design for everything that needs to get done, and often I think it would be useful to have a second person working on the design with me to bounce ideas off of.
Monday, August 4, 2008
NFJS(Mocks, CI, and other tools of Agile Folks)
The first of several Agile talks I went to was Tools to Facilitate Agile Development. The first set of tools was all based around Unit Testing. Most of this was about Mock objects and Junit in general. We already use EasyMock and try to do some testing ( we are getting better all the time ). One Unit testing tool I had never heard of, and plan to explore more is JUnitPerf. One of our big problems at work is that our Jboss server PermGens A LOT. We have never been able to reproduce this in tests. Maybe using the load testing stuff here I will be able to reproduce it, then we can finally start debugging it. There was some good information on Measuring Design Quality and trying to get you're code closer to the Main Sequence...a good balance between abstractness and concrete implementation that actually does something. I like this, but as we try to introduce more tools and techniques I think this one is going to end up down on the list. We have been looking at Code Coverage tools on our own already and I think this is one of the things that we will try to implement pretty soon. Maybe not on everything, but at least on some stuff. I had never heard of, or much less thought about, using a tool like Jester that attempts to figure out of if you are writing "good" tests by modifying your code at run time to verify that if it changes the code your tests will start to fail. I like this, at least for now, as something that we should encourage people to run on their own while writing tests. One of the things Venkat mentioned about Jester is its ability to help you learn how to write good tests. But it also throws a lot of false positives, so I'm not sure I'm ready to throw that on Cruise Control just yet. I have run PMD many times on my own code, and I like it, but it also throws a lot of false positives, so until we get better Junit coverage, and eliminate the errors generated by FindBugs then I'm not sure I'm ready to jump on PMD or its CPD tool. As it is today we do not run FindBugs on our projects, except for the couple that I am working on. Mostly the reason so far has been that there are too many bugs reported and "why should we fix those before we fix the ones that the customer is pointing out to us" I'll continue to fight back on that one as I think we should get Find Bugs running on everything ASAP. Venkat also touched on CI, and over the last year we have gotten Cruise Control up and running, and within the last 3-4 months we now have all of our projects in Cruise Control and running the unit tests when they exist.
I also went to Venkat's talk Mocking to Facilitate Unit Testing. This was another talk where I have already been doing this for a while, and have introduced it to the rest of the team. I really went to this talk because I had introduced myself to EasyMock and taught myself how to use the framework, so I wanted to know if we were doing things right. As it turns out we are on the right track. The one thing that I took from this class was...”don't use a Mock Framework just for the sake of using a Mock Framework”. I'm not going to go back to my old code and change the tests, but I know from know on I'll really consider heavily whether or not I really need a mock object or if I can create a better one on my own. And as I look back on it, I really think that I will tend to lean on EasyMock more often than not because I like what it offers. The other thing that came from this talk ( at least I think it was this one ) was the idea of introducing Groovy to the company by writing unit tests in Groovy. This would offer a lot of the mock stuff right inside of it.
Jeff Brown gave a Test Driven Development with Groovy and Grails talk. There was actually not a lot here that I had not seen before, which kinda surprised me a bit. There were really two things that I took aways from this. First that I like TDD with a dynamic language because you can write a set of tests for stuff that doesn't exist and still run them. For Example...a class is going to have 3 methods that interact with each other. I can write up the unit tests and run them before the class is implemented. In Java you could write up the tests, but you definitely can't run them because the code won't compile. This is a pretty minor thing, but I really like that idea. The other was the coolness of the Expando class, because of Duck Typing you can build and Expando and use it as a mock for anything...really cool. Interestingly not a groovy specific thing, but he talked a bit about Canoo Webtest for testing grails applications, and I think we could really utilize that to test our Web stuff since today we do NO tests on our front ends. I think I will give that a try the next time I edit the front end stuff, which hopefully will be a long time from now.
I also went to Venkat's talk Mocking to Facilitate Unit Testing. This was another talk where I have already been doing this for a while, and have introduced it to the rest of the team. I really went to this talk because I had introduced myself to EasyMock and taught myself how to use the framework, so I wanted to know if we were doing things right. As it turns out we are on the right track. The one thing that I took from this class was...”don't use a Mock Framework just for the sake of using a Mock Framework”. I'm not going to go back to my old code and change the tests, but I know from know on I'll really consider heavily whether or not I really need a mock object or if I can create a better one on my own. And as I look back on it, I really think that I will tend to lean on EasyMock more often than not because I like what it offers. The other thing that came from this talk ( at least I think it was this one ) was the idea of introducing Groovy to the company by writing unit tests in Groovy. This would offer a lot of the mock stuff right inside of it.
Jeff Brown gave a Test Driven Development with Groovy and Grails talk. There was actually not a lot here that I had not seen before, which kinda surprised me a bit. There were really two things that I took aways from this. First that I like TDD with a dynamic language because you can write a set of tests for stuff that doesn't exist and still run them. For Example...a class is going to have 3 methods that interact with each other. I can write up the unit tests and run them before the class is implemented. In Java you could write up the tests, but you definitely can't run them because the code won't compile. This is a pretty minor thing, but I really like that idea. The other was the coolness of the Expando class, because of Duck Typing you can build and Expando and use it as a mock for anything...really cool. Interestingly not a groovy specific thing, but he talked a bit about Canoo Webtest for testing grails applications, and I think we could really utilize that to test our Web stuff since today we do NO tests on our front ends. I think I will give that a try the next time I edit the front end stuff, which hopefully will be a long time from now.
NFJS ( Groovy )
In a word "awesome". I think this really comes down more to dynamic languages in General. I went to several of Jeff Browns talks on Groovy, and along the way I heard mention several times that this stuff can be done in most, if not all, dynamic languages.
One of the things I like about groovy is the fact that it interacts so well with Java. There are a couple of reasons that I really like this. 1. I'm coming from nothing but Java ( and a little Cobol ) over the past few years so it is really easy for me to transition to the new syntax. 2. We built a base class "application" that we utilize to drive all of our batch processing stuff, since Groovy can work with Java so well I can write an "application" class that extends this base application and still take advantage of all the functionality that we have already built.
I went the Thorough Introduction to Groovy talk even though I had written some Groovy before, it was all self taught so I saw this as a good opportunity to find out if I was doing it "right". I was happy to find out that I am definitely on the right track. I never knew about the Builders before, but I sure wish that I had because I wrote a whole lot of Java code to mimic that kind of simplicity for a project last year. Of course its not at all the same, because Groovy uses some meta programming to capture method calls to methods that don't exist to get the Job done, but I think my solution makes writing XML just as easy as the Builder does. Of course the big drawback is that I spent a few months writing the code and I can almost guarantee there are at least a few bugs in it because we just made it work for exactly our use case. The other thing in the intro that I wish I would have known about earlier is the Groovy Console. That is really handy for playing around with the syntax to figure out how to do things. I had previously kinda knew what Groovy Properties were, but since I never used them before I did not realize how powerful they could be in respect to less coding, and more importantly the ability to call constructors with a map of what properties you want set. I can't count the number of times I wish a particular domain object had a constructor with a given set of parameters when I was building my unit tests. One thing that I really wish more "intro" class/talks/websites would do different, and this one fell into the same trap but with a warning up front, is that closers are too often associated with loops. Jeff did go through a few other examples like the number.times method that accepts, but that can be done in Java with a for loop. I really like closures in that you can write more concise/cleaner code with them, but I'm still searching for a good example of something that would be just outright painful, or better yet impossible, to do in Java that a closure can do easily. Thinking about them it just feels like that example must be out there. There was just enough of a hint at Meta Programming here to get me to come back to the Meta Programming class later on, so I'll talk more about Meta programming with that class.
One of the things I like about groovy is the fact that it interacts so well with Java. There are a couple of reasons that I really like this. 1. I'm coming from nothing but Java ( and a little Cobol ) over the past few years so it is really easy for me to transition to the new syntax. 2. We built a base class "application" that we utilize to drive all of our batch processing stuff, since Groovy can work with Java so well I can write an "application" class that extends this base application and still take advantage of all the functionality that we have already built.
I went the Thorough Introduction to Groovy talk even though I had written some Groovy before, it was all self taught so I saw this as a good opportunity to find out if I was doing it "right". I was happy to find out that I am definitely on the right track. I never knew about the Builders before, but I sure wish that I had because I wrote a whole lot of Java code to mimic that kind of simplicity for a project last year. Of course its not at all the same, because Groovy uses some meta programming to capture method calls to methods that don't exist to get the Job done, but I think my solution makes writing XML just as easy as the Builder does. Of course the big drawback is that I spent a few months writing the code and I can almost guarantee there are at least a few bugs in it because we just made it work for exactly our use case. The other thing in the intro that I wish I would have known about earlier is the Groovy Console. That is really handy for playing around with the syntax to figure out how to do things. I had previously kinda knew what Groovy Properties were, but since I never used them before I did not realize how powerful they could be in respect to less coding, and more importantly the ability to call constructors with a map of what properties you want set. I can't count the number of times I wish a particular domain object had a constructor with a given set of parameters when I was building my unit tests. One thing that I really wish more "intro" class/talks/websites would do different, and this one fell into the same trap but with a warning up front, is that closers are too often associated with loops. Jeff did go through a few other examples like the number.times method that accepts, but that can be done in Java with a for loop. I really like closures in that you can write more concise/cleaner code with them, but I'm still searching for a good example of something that would be just outright painful, or better yet impossible, to do in Java that a closure can do easily. Thinking about them it just feels like that example must be out there. There was just enough of a hint at Meta Programming here to get me to come back to the Meta Programming class later on, so I'll talk more about Meta programming with that class.
Meta Programming is Awesome. Jeff Brown gave a pretty cool talk on this stuff. This is one of those topics that I really never heard of and was totally blown away by how useful it is. It allows you to intercept any method call that you want to ( like AOP ) and do stuff. The big thing that standard AOP doesn't let you do is you can call methods that doen't exist and use Meta Programming to do something useful. The best example Jeff gave was the XML MarkupBuilder. Also using Closures to to add functionality to classes, as well as the more traditional MetaClass way of adding functionality to classes.
No Fluff Just Suff
I spent the weekend along with a bunch of guys from our team at the NFJS show in Green Bay: www.nofluffjuststuff.com. Overall I think the conference was very good. One of the big things that I took from the Conference is the idea of having a retrospective after you finish an iteration. One way to view this weekend was as an iteration of learning, so I'm gonna have my own little retrospective here.
Ok this got really long as I was writting it so I have broken it into several smaller enteries. And just a quick summary here.
10 Best things from the weekend:
Instead of simply writting unit tests I need to take a bigger picutre and make sure I write good integration tests as well.
The biggest thing that could impove the team..
Doing peer code reviews before every check in would help both the reviewer and the reviewe.
Update:
Here are links for the 3 detail entries...
NFJS ( Groovy )
NFJS(Mocks, CI, and other tools of Agile Folks)
NFSJ(TDD, DDT, and other practices of Agile Folk)
Ok this got really long as I was writting it so I have broken it into several smaller enteries. And just a quick summary here.
10 Best things from the weekend:
- Integration Tests are equally as important as unit tests
- Peer Code review should be done before EVERY Checkin
- Peer code review does NOT have to be done with somebody that is an expert in the code you are working on
- DDT - Defect Driven Testing ( if there is a bug write a test before fixing the bug )
- Groovy is well groovy
- Meta Programming is Extreamly Powerful
- You can manipulate Java objects at runtime from groovy using Meta Programming
- Code Coverage should include all Unit and Integration/Functional Tests and System testing
- Work in smaller iterations ( typically 1-4 weeks )
- Within iterations break tasks down into things that can be completed in 0.5-3 days
Instead of simply writting unit tests I need to take a bigger picutre and make sure I write good integration tests as well.
The biggest thing that could impove the team..
Doing peer code reviews before every check in would help both the reviewer and the reviewe.
Update:
Here are links for the 3 detail entries...
NFJS ( Groovy )
NFJS(Mocks, CI, and other tools of Agile Folks)
NFSJ(TDD, DDT, and other practices of Agile Folk)
Introduction
For those of you who have been reading my personal blog ( edvedafi.blogspot.com ) this will be pretty much the opposite. In here I intend to keep this strictly Tech related and keep the other blog strictly personal/family related. So if you interested in the latest crazy thing that Kevin and Alex have been up to you probably want the other blog, and if your interested in techy/geeky stuff you've come to the right place.
I am by trade a Java Developer so mostly this will focus on Java stuff. However I have recently learned some Groovy. At work its pretty much just Java batch processing along with an occasional bash script, Jasper report, and on rare occasions I might dig just a little bit into our JSF website. Over the course of the last year we have started to implement a bunch of "agile" techniques. I have been a driving force behind getting the ideas bubbling in my managers. So a lot of this blog will be about how we go about implementing the techniques.
I also have been working on a project on my own to build a website for managing/selling/trading Collection of ( insert collectable product here ) . This all is driven by the fact that I cannot find anything websites out there ( including Beckett's My Collections ) that allow me to do what I want ot track my Sports cards & memoriabila collection. A couple have been close but all lack features. As a long ( very long ) term goal I'd like to build up enough of a user base to be able to build in an algorithm that helps the user determine high and low figures for what they might expect to buy/sell cards for. Most importantly though is that I want to get just a good collection management tool out there. At this point I'm thinking that I might use EJB3 as the backend, mostly because that is what we use at work and I like it, but I might try a different approach just to do something new. I'm leaning towards writing the front end in Groovy, which leads me to think that I might just go all Grails and get ride of the ejb layer, or I may go Grails and connect it up to the EJBs and/or webservices. Lots of ideas floating around in my head. But this is a project that has been floating around in there for several years and very very little code is getting written on it.
Jason
I am by trade a Java Developer so mostly this will focus on Java stuff. However I have recently learned some Groovy. At work its pretty much just Java batch processing along with an occasional bash script, Jasper report, and on rare occasions I might dig just a little bit into our JSF website. Over the course of the last year we have started to implement a bunch of "agile" techniques. I have been a driving force behind getting the ideas bubbling in my managers. So a lot of this blog will be about how we go about implementing the techniques.
I also have been working on a project on my own to build a website for managing/selling/trading Collection of ( insert collectable product here ) . This all is driven by the fact that I cannot find anything websites out there ( including Beckett's My Collections ) that allow me to do what I want ot track my Sports cards & memoriabila collection. A couple have been close but all lack features. As a long ( very long ) term goal I'd like to build up enough of a user base to be able to build in an algorithm that helps the user determine high and low figures for what they might expect to buy/sell cards for. Most importantly though is that I want to get just a good collection management tool out there. At this point I'm thinking that I might use EJB3 as the backend, mostly because that is what we use at work and I like it, but I might try a different approach just to do something new. I'm leaning towards writing the front end in Groovy, which leads me to think that I might just go all Grails and get ride of the ejb layer, or I may go Grails and connect it up to the EJBs and/or webservices. Lots of ideas floating around in my head. But this is a project that has been floating around in there for several years and very very little code is getting written on it.
Jason
Subscribe to:
Posts (Atom)
