how to find a search term in source code

Question

I'm looking for a way to search for a given term in a project's C/C++ code, while ignoring any occurrences in comments and strings.

As the code base is rather large, i am searching for a way to automatically identify the lines of code matching my search term, as they need manual inspection.

If possible I'd like to perform the search on my linux system.

background

the code base in question is a realtime signal processing engine with a large number of 3rd party plugins. plugins are implemented in a variety of languages (mostly C, but also C++ and others; currently I only care for those two), no standards have been enforced.

our code base currently uses the built-in type float for floating-point numbers and we would like to replace that with a typedef that would allow us to use doubles. we would like to find all occurrences of float in the actual code (ignoring legit uses in comments and printouts).

What complicates things furthermore, is that there are some (albeit few) legit uses of float in the code payload (so we are really looking for a way to identify all places that require manual inspection, rather than run some automatic search-and-replace.)

the code also contains C-style static casts to (float), so relying on compiler warnings to identify type mismatches is often not an option.

the code base consists of more than 3000 (C and C++) files accumulating about 750000 lines of code.

the code is cross-platform (linux, osx, w32 being the main targets; but also freebsd and similar), and is compiled with the various native compilers (gcc/g++, clang/clang++, VisualStudio,...).

so far...

so far I'm using something ugly like:

 grep "\bfloat\b" | sed -e 's|//.*||' -e 's|"[^"]*"||g' | grep "\bfloat\b"

but I'm thinking that there must be some better way to search only payload code.

Answer 1

IMHO there is a good answers on a similar question at "Unix & Linux":

grep works on pure text and does not know anything about the underlying syntax of your C program. Therefore, in order not search inside comments you have several options:

1. Strip C-comments before the search, you can do this using gcc -fpreprocessed -dD -E yourfile.c For details, please see Remove comments from C/C++ code

2. Write/use some hacky half-working scripts like you have already found (e.g. they work by skipping lines starting with // or /*) in order to handle the details of all possible C/C++ comments (again, see the previous link for some scary testcases). Then you still may have false positives, but you do not have to preprocess anything.

3. Use more advanced tools for doing "semantic search" in the code. I have found "coccigrep": http://home.regit.org/software/coccigrep/ This kind of tools allows search for some specific language statements (i.e. an update of a structure with given name) and certainly they drop the comments.

https://unix.stackexchange.com/a/33136/158220

Although it doesn't completely cover your "not in strings" requirement.

Answer 2

It might practically depend upon the size of your code base, and perhaps also on the editor you are usually using. I am suggesting to use GNU emacs (if possible on Linux with a recent GCC compiler...)

For a small to medium size code (e.g. less than 300KLOC), I would suggest using the grep mode of Emacs. Then (assuming you have bound the next-error Emacs function to some key, perhaps with (global-set-key [f10] 'next-error) in your ~/.emacs...) you can quickly scan every occurrence of float (even inside strings or comments, but you'll skip very quickly such occurrences...). In a few hours you'll be done with a medium sized source code (and that is quicker than learning how to use a new tool).

For a large sized code (millions of lines), it might be worthwhile to customize some static analysis tool or compiler. You could use GCC MELT to customize your GCC compiler on Linux. Its findgimple mode could be inspirational, and perhaps even useful (you probably want to find all Gimple assignments targeting a float)

BTW, you probably don't want to replace all occurrences -but only most of them- of the float type with double (probably suitably typedef-ed...), because very probably you are using some external (or standard) functions requiring a float.

The CADNA tool might also be useful, to help you estimate the precision of results (so help you deciding when using double is sensible).

Using semantical tools like GCC MELT, CADNA, Coccinelle, Frama-C (or perhaps Fluctuat, or Coccigrep mentioned in g0hl1n's answer) would give more precise or relevant results, at the expense of having to spend more time (perhaps days!) in learning and customizing the tool.