[Tutor] Integrating TDD into my current project work-flows

[WolfRage]

I would like some help integrating TDD into my current projects.
My chosen TDD framework is unittest from the standard library.
My system details are: Linux Mint 17.1 64-bit, Python 3.4, bzr(for
version control).

My projects are structured like:
Project > develop > Project > Project > __main__.py
tests > __main__.py
I want to be able to execute my Project from a cwd of:
Project/develop/Project
as: Python3 -m Project
That currently works.
But executing my tests as: Python3 -m tests
requires that test_Project.py has this hack to import the Project module:
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
do you consider that OK? Is there a better way?
I call it a hack because every testcase file will have to have this line
added to it in order to work.

I am also using coverage.py and it is good but seems to be testing the
coverage of my tests, which is not desired, how can I stop this
behaviour. Or is it really OK.

Output of running the tests:
python3 -m tests
Ran Main.
.
----------------------------------------------------------------------
Ran 1 test in 0.001s

OK
Name Stmts Miss Cover Missing
--------------------------------------------------
Project/Project 3 0 100%
tests/test_Project 8 0 100%
--------------------------------------------------
TOTAL 11 0 100%


The test files:
{FileName = __main__.py}
# -*- coding: utf-8 -*-
if __name__ == '__main__':
import coverage
import unittest
cov = coverage.coverage()
cov.start()
# .. call your code ..
from .test_Project import ProjectTestCase # lint:ok
unittest.main(exit=False)
cov.stop()
cov.save()
import sys
cov.report(file=sys.stdout)


{FileName = test_Project.py}
# -*- coding: utf-8 -*-
import os
import sys
import unittest
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
from Project import Project


class ProjectTestCase(unittest.TestCase):

def test_example(self):
self.assertTrue(Project.main())


The Project Files:
{FileName = __main__.py}
# -*- coding: utf-8 -*-

if __name__ == '__main__':
from . import Project
Project.main()

{FileName = Project.py}
# -*- coding: utf-8 -*-


def main():
return True
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Martin A. Brown]

Hi there,

Post by WolfRage
I would like some help integrating TDD into my current projects.
My chosen TDD framework is unittest from the standard library. My
system details are: Linux Mint 17.1 64-bit, Python 3.4, bzr(for
version control).
Project > develop > Project > Project > __main__.py
tests > __main__.py
Project/develop/Project
as: Python3 -m Project
That currently works.
But executing my tests as: Python3 -m tests
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))

Yes, a bit ugly. Have you tried using nose? I have used a
similar project development tree and use nose to run the tests.

Here are a few sample command-lines (I'm on an opensuse-13.2 system
with Python 3.4 and nose for Python-3.4, which is called
'nosetests-3.4'):

nosetests-3.4 -- ./tests/
nosetests-3.4 --with-coverage -- ./tests/
nosetests-3.4 --with-coverage --cover-package=Project -- ./tests/

I like nose because it will discover any unittest and doctest
testing code under the specified files/directories.

Post by WolfRage
do you consider that OK? Is there a better way?
I call it a hack because every testcase file will have to have
this line added to it in order to work.

Agreed, is a hack.

Post by WolfRage
I am also using coverage.py and it is good but seems to be testing
the coverage of my tests, which is not desired, how can I stop
this behaviour. Or is it really OK.

That's precisely what coverage is supposed to do--that is it should
report on how much of the 'code under test' has been exercised by
the testing code. So, in fact, it's better than OK--it's the
primary point!

There are two good things about using coverage.

#1: You see how much more effort you should invest to get
substantial testing coverage of the code under test.
[Though, some code is easy to test and others very difficult.]

#2: You get a report of the lines in the code under test which are
NOT yet tested; handy!

Good luck,

-Martin

[WolfRage]

Post by Martin A. Brown
Hi there,
Yes, a bit ugly. Have you tried using nose? I have used a similar
project development tree and use nose to run the tests.

I had thought about it, but decided not too, since it was not part of
the standard library. But then again, it is not like I don't know how to
install additional dependencies for my projects.

Post by Martin A. Brown
Here are a few sample command-lines (I'm on an opensuse-13.2 system with
nosetests-3.4 -- ./tests/
nosetests-3.4 --with-coverage -- ./tests/
nosetests-3.4 --with-coverage --cover-package=Project -- ./tests/
I like nose because it will discover any unittest and doctest testing
code under the specified files/directories.

It looks pretty easy to use, does it have an API like coverage and
unittest so that I can write my test scripts and simply execute them
rather than running commands from the terminal? I find that scripts
prevent errors as compared to typing commands on the terminal.
<SNIP>

Post by Martin A. Brown
-Martin

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Oscar Benjamin]

Post by WolfRage
I would like some help integrating TDD into my current projects.
My chosen TDD framework is unittest from the standard library.
My system details are: Linux Mint 17.1 64-bit, Python 3.4, bzr(for version
control).
Project > develop > Project > Project > __main__.py
tests > __main__.py
Project/develop/Project
as: Python3 -m Project
That currently works.

That will only work if there is also a file __init__.py under the
Project/develop/Project/Project directory (alongside the __main__.py
file). Adding a __main__.py file allows the directory to be treated as
a Python script e.g.:

$ python3 Project/develop/Project/Project
$ python3 Project # Assuming cwd is Project/develop/Project

The -m switch searches for a script as if it were a module using the
module search path. In this case the directory containing the package
Project must be on sys.path and one way to achieve this is to change
directory to the Project/develop/Project directory. Then the Project
folder in this directory is on sys.path.

However a directory is not considered a package unless it contains a
file __init__.py. So if the directory is on sys.path (e.g. in the cwd)
AND their is an __init__.py file then you can do

$ python3 -m Project

and the code in __main__.py will run.

Assuming you have the same setup in the
Project/develop/Project/Project/tests directory (both __init__.py and
__main__.py and Project is on sys.path) then you can run
tests/__main__.py using the module search path as

$ python3 -m Project.tests

Note that this is the same name that you would use to import from the
tests package in interpreter e.g.

Post by WolfRage

from Project.tests import test_stuff

imports from the __init__.py in the tests folder.

Post by WolfRage
But executing my tests as: Python3 -m tests
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
do you consider that OK? Is there a better way?
I call it a hack because every testcase file will have to have this line
added to it in order to work.

I'm not 100% sure I understand what you're doing but hopefully what
I've written above leads to a better solution.

Post by WolfRage
I am also using coverage.py and it is good but seems to be testing the
coverage of my tests, which is not desired, how can I stop this behaviour.
Or is it really OK.

I don't see any problem with it since it shows how many of your tests
are being run. If you're not running all of your tests then that's a
problem. I don't know how involved your projects are but often large
projects will have tests that run conditionally for example they
only/don't run on certain platforms. In that case it could be useful
each time you run your tests to get a loose idea of how many tests are
being skipped.

Of course it may just be a distraction from the main goal which is
ensuring good coverage of the exported code. You can control which
files coverage tracks and reports the coverage of. I don't know how to
do it the way that you're running coverage (from within Python code).
I've always run coverage from the command line interface using a
configuration file. See here for more on that:

http://nedbatchelder.com/code/coverage/cmd.html#cmd
http://nedbatchelder.com/code/coverage/config.html#config
http://nedbatchelder.com/code/coverage/source.html#source


--
Oscar
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[WolfRage]

On 05/05/2015 10:59 AM, Oscar Benjamin wrote:
<SNIP>

Post by Oscar Benjamin

Post by WolfRage
Project > develop > Project > Project > __main__.py
tests > __main__.py
Project/develop/Project
as: Python3 -m Project
That currently works.

That will only work if there is also a file __init__.py under the
Project/develop/Project/Project directory (alongside the __main__.py
file). Adding a __main__.py file allows the directory to be treated as

But actually there is no __init__.py in Project/develop/Project/Project/
There is only __main__.py and Project.py files. The __main__.py file
imports Project.py.

Post by Oscar Benjamin
$ python3 Project/develop/Project/Project
$ python3 Project # Assuming cwd is Project/develop/Project

I tested the above on the terminal and it does not work. I get:
***@wolfrage-Lemur-UltraThin
~/HomePlusWolfrage/Projects/Project/develop/Project $ python3 Project
Traceback (most recent call last):
File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "Project/__main__.py", line 4, in <module>
from . import Project
SystemError: Parent module '' not loaded, cannot perform relative import

Post by Oscar Benjamin
The -m switch searches for a script as if it were a module using the
module search path. In this case the directory containing the package
Project must be on sys.path and one way to achieve this is to change
directory to the Project/develop/Project directory. Then the Project
folder in this directory is on sys.path.

Did you perhaps mean the Project/develop/Project/Project/ directory?

Post by Oscar Benjamin
However a directory is not considered a package unless it contains a
file __init__.py. So if the directory is on sys.path (e.g. in the cwd)
AND their is an __init__.py file then you can do
$ python3 -m Project
and the code in __main__.py will run.
Assuming you have the same setup in the
Project/develop/Project/Project/tests directory (both __init__.py and
__main__.py and Project is on sys.path) then you can run
tests/__main__.py using the module search path as

I think I may have confused you. The layout is like this:
Project/develop/Project/Project
Project/develop/Project/tests
Thus tests is not contained in module's directory.
My reasoning for this is that the tests would be shipped with my
production code. I am currently undecided as to whether this is a good
or bad thing. But the more that I read about testing the more it seems
to be a good thing. So perhaps the best solution is to move the tests
into the module's directory. Then the below information would probably work.

Post by Oscar Benjamin
$ python3 -m Project.tests
Note that this is the same name that you would use to import from the
tests package in interpreter e.g.

Post by WolfRage

from Project.tests import test_stuff

imports from the __init__.py in the tests folder.

<SNIP>

Post by Oscar Benjamin
Oscar

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[WolfRage]

Update: My previous hack, has been changed. I now put:

import os
import sys
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))

in the __main__.py file located under the tests/ directory and it is
only needed the one time in that one file. Not sure why I was thinking I
would need to do that for every test. But that now makes the tests
standalone.

Also I understand the benefit of coverage showing the coverage for test
files too, so this is not a problem.

As I am reading about TDD and unittests and the likes. I am thinking
that testing from a "higher" level is better as compared to the "unit"
level testing.
It seems to me that in order to get the benefits of testing I need to
have less test code that will actually test more of my real code. I am
seeing many potential names for this concept (Blackbox Automated
Testing; System Level Test; API Level Tests) and would like your inputs
on it.
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Alan Gauld]

Post by WolfRage
As I am reading about TDD and unittests and the likes. I am thinking
that testing from a "higher" level is better as compared to the "unit"
level testing.

Not better, just necessary. The two concepts are complementary.
You need both. The developer primarily needs unit testing, the
integrator*(who may of course be the developer in a different
role) needs integration testing and the client/project manager
needs system testing and acceptance testing. They are all part
of a project (especially big/commercial projects)

Post by WolfRage
It seems to me that in order to get the benefits of testing I need to
have less test code that will actually test more of my real code. I am
seeing many potential names for this concept (Blackbox Automated
Testing; System Level Test; API Level Tests) and would like your inputs
on it.

Don't underestimate the scale of testing. It is not unusual to
have more test code than functional code! (although it is still
the exception!) In real-world commercial projects testing (and
the associated debugging) typically consumes about 25-40% of
the total project budget. Baseline coding by contrast is only
about 10-25%, sometimes much less.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[WolfRage]

On 05/05/2015 06:49 PM, Alan Gauld wrote:
<SNIP>

Post by Alan Gauld
Not better, just necessary. The two concepts are complementary.
You need both. The developer primarily needs unit testing, the
integrator*(who may of course be the developer in a different
role) needs integration testing and the client/project manager
needs system testing and acceptance testing. They are all part
of a project (especially big/commercial projects)

So how low level of unit testing is acceptable. My use case is the sole
programmer on a team project. Their is potential for another programmer
to join the ranks but that has not happened yet. Additionally the
project is well under way. But Feature additions have slowed so I want
to make the code less buggy, and so I am hoping to re-factor my code
now. I think some unit tests could improve the end result of my
re-factoring, but where to start is the toughest problem for me to
solve. Especially given the time trade off that is at least initially
required for unit tests.
<SNIP>

Post by Alan Gauld
Don't underestimate the scale of testing. It is not unusual to
have more test code than functional code! (although it is still
the exception!) In real-world commercial projects testing (and
the associated debugging) typically consumes about 25-40% of
the total project budget. Baseline coding by contrast is only
about 10-25%, sometimes much less.

I agree I have seen this sort of budget numbers in the government
project that I have been involved in. Especially once the project hits a
more general life cycle/maintenance mode.

So how can I make unit testing apply to my project without starting from
scratch? And how low should I test(ie. every function; every class;
every interface; system level)?
Thank you for any insight and all of your help.
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Mark Lawrence]

Post by WolfRage
<SNIP>

Post by Alan Gauld
Not better, just necessary. The two concepts are complementary.
You need both. The developer primarily needs unit testing, the
integrator*(who may of course be the developer in a different
role) needs integration testing and the client/project manager
needs system testing and acceptance testing. They are all part
of a project (especially big/commercial projects)

So how low level of unit testing is acceptable. My use case is the sole
programmer on a team project. Their is potential for another programmer
to join the ranks but that has not happened yet. Additionally the
project is well under way. But Feature additions have slowed so I want
to make the code less buggy, and so I am hoping to re-factor my code
now. I think some unit tests could improve the end result of my
re-factoring, but where to start is the toughest problem for me to
solve. Especially given the time trade off that is at least initially
required for unit tests.
<SNIP>

Post by Alan Gauld
Don't underestimate the scale of testing. It is not unusual to
have more test code than functional code! (although it is still
the exception!) In real-world commercial projects testing (and
the associated debugging) typically consumes about 25-40% of
the total project budget. Baseline coding by contrast is only
about 10-25%, sometimes much less.

I agree I have seen this sort of budget numbers in the government
project that I have been involved in. Especially once the project hits a
more general life cycle/maintenance mode.

If and only if the project ever gets this far. One third of major
projects fail to deliver anything. Part of the reason for that is
growing bug lists owing to limited or even no (controlled) testing.

Post by WolfRage
So how can I make unit testing apply to my project without starting from
scratch? And how low should I test(ie. every function; every class;
every interface; system level)?
Thank you for any insight and all of your help.

Ensure that every publicly available part of your code is tested. Don't
bother with internal helper functions or similar. Remember that the
toughest parts to test are often the error handling capabilities and the
rarely occurring edge cases, so up front thought can save a huge amount
of time, money and effort later on.
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[WolfRage]

I find myself in the same mind set as this individual:
http://stackoverflow.com/a/64453/4285911
It is hard to write a proper test with out me initially outlining where
I am going. Perhaps I need to better understand planning and drafting a
programming project before I can hope to emulate TDD.

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Thomas C. Hicks]

Post by WolfRage
http://stackoverflow.com/a/64453/4285911
It is hard to write a proper test with out me initially outlining
where I am going. Perhaps I need to better understand planning and
drafting a programming project before I can hope to emulate TDD.

For what it is worth the idea of user stories really helped me develop
an approach to testing (or should I say, start an approach to testing).
The tutorial (here
<http://chimera.labs.oreilly.com/books/1234000000754/index.html>)
covering Django development really drove that idea and its
implementation home for me. I believe the user story idea first shows
up in chapter 2 of the book.

thomas
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor