PyPI 官网下载 | doit-0.17.0.tar.gz

========================= DoIt - a build-tool tale ========================= DoIt is a built-tool like written in python. In this post explain my motivation for writting yet another buil tool. If you just want to use it. Please check the `website <http://python-doit.sourceforge.net/>`_ Build-tool ---------- I started working on a web project... As a good TDD disciple I have lots of tests spawning everywhere. There are plain python unittest, twisted's trial tests and Django specific unit tests. That's all for python, but I also have unit tests for javascript (using a home grown unit test framework) and regression tests using Selenium. Running lint tools (`JavaScriptLint <http://www.javascriptlint.com/>`_ and `PyFlakes <http://divmod.org/trac/wiki/DivmodPyflakes>`_) are as important. So I have seven tools to help me keeping the project healthy. But I need one more to control the seven tools! Actually there are more. I am not counting the javascript compression tool, the documentation generator... I am not looking for a continuous integration (at least right now). I want to execute the tests in a efficient way and get problems before committing the code to a VCS. | - What tool do we use to automate running tasks? | - GNU Make. Or any other build tool. SCons ----- I had the misfortune to (try to) debug some `Makefile <http://www.gnu.org/software/make/>`_'s before. XML based was never really an `option <http://ant.apache.org/>`_ to me. Since I work with python `SCons <http://www.scons.org/>`_ looked like a good bet. SCons. Writing the rules/tasks in python helps a lot. But the configuration (construct) file is not as simple and intuitive as I would expect. Maybe too powerful for my needs. Thats ok I don't have to write new "Builders" that often. Things went ok for a while... but things started to get too slow. Normal python tests are fast enough not to bother about it. But Django tests using postgres execution time do bother. The javascript tests run on the browser. So it needs to start the server, launch the browser, load and execute the tests... uuoooohhhh. Most of the time i *really* need to execute just a subset of tests/tasks. The whole point of build tools is to keep track of dependencies and re-build only what is necessary, right? The problem with tests is that actually i am not *building* anything. I am executing tasks(in this case tests). Building something is a "task" with a "target" file(s), but running a test is a "task" with no "target". The problem is that build tools were designed to keep track of target/file dependencies not task dependencies. Yes I know you can `use <http://www.gnu.org/software/make/manual/make.html#Empty-Targets>`_ `hacks <http://martinfowler.com/bliki/OutputBuildTarget.html>`_ to pretend that every task has a target file. But I was not really willing to do this... I was not using any of the great SCons features. Actually at some point I easily substitute it to a simple (but lengthy) python script using the `subprocess <http://docs.python.org/lib/module-subprocess.html>`_ module. Of course this didn't solve the speed problem. DoIt ---- `DoIt <http://python-doit.sourceforge.net/>`_. I want a tool to automatically execute any kind of tasks, having a target or not. It must keep track of the dependencies and re-do (or re-execute) tasks only if necessary, (like every build tool do for target files). And of course it shouldn't get on my way while specifying the tasks. Requirements: . keep track of dependencies. but they must be specified by user, no automatic dependency analysis. (i.e. nearly every build tool supports this) . easy to create new task rules. (i.e. write them in python) . get out of your way, avoid boiler-plate code. (i.e. something like what nose does to unittest) . dependencies by tasks not on files/targets. The only distinctive requirement is item 4. I guess any tool that implements dependency on targets could support dependency on tasks with not so much effort. You just need to save the signature of the dependent files on successful completion of the task. If none of the dependencies changes the signature the task doesn't need to be executed again. Since it is not required to have a target the tasks needs to be uniquely identified. But thats an implementation detail... So how does it look like? extracted from the `tutorial <http://python-doit.sourceforge.net/tutorial.html>`_: Compressing javascript files, and combining them in a single file. I will use `shrinksafe <http://svn.dojotoolkit.org/branches/1.1/util/shrinksafe/custom_rhino.jar>`_. ``compressjs.py`` :: """ dodo file - compress javascript files """ import os jsPath = "./" jsFiles = ["file1.js", "file2.js"] sourceFiles = [jsPath + f for f in jsFiles] compressedFiles = [jsPath + "build/" + f + ".compressed" for f in jsFiles] def create_folder(path): """Create folder given by "path" if it doesnt exist""" if not os.path.exists(path): os.mkdir(path) return True def task_create_build_folder(): buildFolder = jsPath + "build" return {'action':create_folder, 'args': (buildFolder,) } def task_shrink_js(): for jsFile,compFile in zip(sourceFiles,compressedFiles): action = 'java -jar custom_rhino.jar -c %s > %s'% (jsFile, compFile) yield {'action':action, 'name':jsFile, 'dependencies':(":create_build_folder", jsFile,), 'targets':(compFile,) } def task_pack_js(): output = jsPath + 'compressed.js' input = compressedFiles action = "cat %s > %s"% (" ".join(input), output) return {'action': action, 'dependencies': input, 'targets':[output]} Running:: doit -f compressjs.py Let's start from the end. ``task_pack_js`` will combine all compressed javascript files into a single file. ``task_shrink_js`` compress a single javascript file and save the result in the "build" folder. ``task_create_build_folder`` is used to create a *build* folder to store the compressed javascript files (if the folder doesnt exist yet). Note that this task will always be execute because it doesnt have dependencies. But even it is a dependency for every "shrink_js" task it will be executed only once per DoIt run. The same task is never executed twice. And that's all