Opening the Toolbox
Every developer (and in particular) every Web Developer should have several tools in their toolbox.
I’ll reserve #1 for their assorted Web Browsers + devices, and accept #2 for their favorite text editor (Sublime, HippoEDIT, Notepad++, etc.) but that leaves us with the #3 slot open.
I think this is where every developers next tool depends more on their personal approach to development and what role they are filling that day etc. If I’m doing graphics today then PhotoShop is my next weapon, Schema refactoring? then that will be my DB management tool… or based on how long it stays open when we are coding maybe its our console(s)?
However there is one tool I think is really important to developers that gets overlooked and that’s because you typically have to write this tool yourself – and its different for everyone!
Welcome to Searching your code base!
Search & Replace with our friend Regular Expressions
On the surface this is a super easy concept… you want to:
- Find all occurrences
- of ${someValue}
- in files of type ${theseFileTypes}
So you put your grep fu to work using whatever search tools you like, possibly with a funky regular expression to find that dynamic/variable value that you are looking for. It works great, you find everything you’re looking for… right?… wrong!
Sometimes you can’t find what you’re looking for because basic searching is quite limited and as powerful as they are, Regular Expressions have (IMHO) a severe limitation in that you can’t search for: (find this)(but not this word)(followed by this)
You can abuse a RegEx to find matches that don’t contain a word… (great StackOverflow.com example answer) however you will always want to find stuff before/after it and thus it all starts to fall apart.
Other things that aren’t simple to find:
- All files containing ${keywordA} but not ${keywordB}
- All files containing ${keywordA} somewhere in the file after ${keywordB}
- All files containing ${keywordA} without ${keywordB} directly following it
In the case of HTML files (regardless what language you use to generate them) there’s things you might want to be able to check for that go well beyond basic searching. How would you do some of these?
- All files that define ID attributes (e.g. <div id=”foo”>) but only those that accidentally generate duplicate IDs… and if so list which IDs are duplicated
- All files that include a specific JavaScript file but don’t contain any calls to the functions defined in it
- All files that include 3 specific components in any order
The solution is both obvious and just painful enough that many developers give up and don’t attempt to solve the problem – heck I was one of them… for years!
The Solution:
The solution is to write code to do your dirty work 😉 the language you use doesn’t matter as long as you can fairly quickly script up what you need including:
- Recursively search your project directory of files (and be able to cap/control iterations)
- Filter by multiple file types
- Search multiple keywords (simple & RegEx)
- Track and compare result findings
- Output some sort of a log of all the findings
Although you might be thinking… what about the ability to make code replacements? You’re right that would be a bonus but if you get too caught up trying to handle that part (which can double your overall effort) you’ll likely give up or risk causing damage to your code files due to a corner case condition you weren’t expecting.
The good news is that omitting the replacement part actually doesn’t matter because once you have a log of files matching your query its easy to load them in your favorite editor and then run search & replace across your open files as needed.
My Solution:
As I said the language you use doesn’t matter. I went with something easy, it works with the tools I already have at hand, & requires no compiling… PHP!
I’ve created several “base” scripts that are easy to extend/edit as needed for any task at hand and whenever I need one I just hit that file’s URL in my browser and let it do all the hard work 😉
I’ve hosted the base files for this up on GitHub in a project called ProjectQuery (under MIT License) feel free to grab them and start building your own collection of custom code searching tools!
Software Design Blog 