I've been through a very deep dive of faceted search inside Sharepoint lately with some very bright resources. We are using the faceted search 2.0 solution from Codeplex and had to modify some of the code to implement this without special characters showing in the results (see title of this blog). :-)
Just know this: core search results in Sharepoint is an estimate, and depending on the circumstances (tagging, keywords, metadata, etc) it could be a rough estimate. You have to page through every page (or trick the URL parameter) to see the actual correct return number. When you build a faceted search solution on top of this estimate, the facets may have inaccurate numbers as well -- and each time you click the facet, it runs a new query which changes the facet results set again (which is again, an estimate).
In a nutshell, the numbers may never add up .... and this may be hard for some to understand (i.e. accountants). But we did manage to get it close by increasing the Results Display property to 10,000 inside the faceted search web part and changing our metadata properties to display only a single value for each property.