How to 'Cheat' on a Survey - Don't Be A Victim!

How to 'Cheat' on a Survey - Don't Be A Victim!

By CWNP On 04/06/2009 - 10 Comments

Because of the flexibility and reporting capabilities built into most site survey applications, individuals can either knowingly, or accidentally, use these features to ‘Cheat’ and make survey data look different than it really is.

OK, I understand the word ‘Cheat’ is a pejorative, and connotes some sort of blatant attempt to break some rules or misrepresent.  Many individuals just didn’t know any better when they applied these methods in their survey reporting.  I’m not implying that these techniques suggest any malfeasance, only ignorance.

As a customer, the best protection against this type of deception is to request not only the paper or PDF report, but the actual data files so you can review and analyze the data directly yourself.

Below are some of the techniques that allow one to modify and present survey data to reflect whatever you might desire.

Be wary of using any of these techniques on your own analysis or in reviewing data presented by other third parties.

Method 1 – Incorrect Signal Propagation Value

One of the easiest ways to save money and time in the data-gathering phase of an RF site survey is to minimize the number of actual data points collected.

It is possible to use site survey software to interpolate a small number of data points and produce a complete ‘Heat Map’ of an entire building.  The software and algorithms will usually allow for this. Just because the software can try and compensate for a poorly conducted survey doesn’t mean you should accept it.

A telltale sign this is being done is to look at the outside ‘arc’ of coverage.  A tight Signal Propagation Assessment (SPA) value will have a tight arc, a large SPA or ‘guess range’ will have a very wide, almost flat, arc. Of course, if you have the actual survey data, you can see what the Signal Propagation Assessment value is set to.

This technique is used to ‘mask’ a survey with not enough data points and/or a survey that is showing ‘white’ areas on the floor plan.  White does not mean there isn’t RF coverage; it just means there isn’t any data to support the heat map ‘painter’ to color that area.

I suggest you always go with an SPA no larger than that suggested by the survey application’s default values for the type of building you are surveying.

Method 2 – Using ‘Auto Contrast’

When configuring AirMagnet Survey, as an example of a site survey application, you can set the color adjustment to a variety of color palettes.  Some of these are easier to see than others.

Note: Some persons with color-blindness have a hard time with the ‘smooth’ color gradients.  There is one specific color option that is easier for them to see.  But I find this very difficult for me to interpret because of the high-contrast changes.

One of the options in the Color Configuration is to set the ‘Auto Contrast Adjustment’ checkbox.  Normally, the top of the color bar is always the same color, and the bottom is always its same color.  Thus the middle is also always the same color.  Thus a -65dBm is always represented by the same color on screen and in reports.

If you turn on ‘Auto Contrast Adjustment’ the top and bottom colors ‘move’ as you raise or lower the limits.  Thus you can make -65dBm appear as any color you’d like.

This is a nice ‘feature’ is you already have a color palette and expectation; you can make AirMagnet Survey match to your expectations. But, in the wrong hands this one tool allows one to make any survey look good or bad. Any color can mean any dB!

The telltale sign this is being used is when the bottom of the color palette, like red for example is NOT showing for -100dB, something higher on the scale.
 

I have seen some companies actually use this technique to ‘standardize’ their reports so the same colors always mean the same dBm.  Unless you are closely attuned to their legend, you’ll be misled on the signal strengths.

I recommend locking on a single color palette, turning off Auto Contrast Adjustment and having consistent universal colors on all your surveys. Green will always mean the same thing; Blue always the same, etc.

Method 3 – Use of ‘Banded’ Color Schemes

Another one of the many Color options typically available is ‘banded colors’.  The use of this color option is sometimes applied to show what is ‘acceptable’ or not.  Above a certain color band is approved, and below is unacceptable.

The problem with this technique is that there is often only a single dB difference between the color bands.  So the strong, high-contrast line between acceptable and unacceptable is only 1 dB… This mis-represents the actual data and how close the subtle changes in RF are.

Smooth color palettes represent these slight changes in RF in a more representative fashion.

Method 4 – Access Points Where No Data Exists

Site survey software and algorithms often compensate for a poorly collected survey by generating ‘phantom’ data based on where an end-user ‘places’ and access point on the survey screen.

It is easy and possible for accidents to take place and an access point is placed not where it is in real life, but where you’d like it to be. Or the opposite, one can place an access point where it is in the real world, but not have any data collection points near that location.

In either case, the calculated and interpolated data is manipulated by the site survey software to generate RF data to support in placement location and power setting of the Access Point in question.

If your data collection is accurate and has followed all of the survey rules, there should be no need to process the data with this access point placement.  If you do need the graphic of the access point on the screen, just place it without processing the data. (Display Only)  When used incorrectly, the results show much stronger signals around the access point than were actually collected during the survey.

Note: If you show the resulting screen with placed-and-processed access points using the 3D display option you’ll easily see these ‘phantom’ results.  I call it the ‘Circus Tent Syndrome’ – each access point has a little volcano looking mountain directly around its center point.

If there are no data capture points (red dots or blue dots) near an access point, but it shows very strong signal, then someone has mis-used this feature.

Method 5 – One-Sided Guesses

No ‘One Way Guesses’. This happens when you don’t capture on both sides of what you care about.  If you take one data capture point on the inside perimeter of your building, and don’t also take one on the outside, then the site survey application doesn’t know anything about the ‘thickness’ (in RF Attenuation) of the exterior wall.  It ‘learns’ this by having data captures on both sides of the wall.

If your survey was more ‘inside out’ the data processing algorithms must assume there is no walls, and uses a free-space-loss calculation. In the absence of better data (both sides of what you care about) – free-space-loss is all the software has to go on.

If you look at your survey results and there is heat map information, but you can see a wall between where the data point was collected, and a location where you ‘care’ – the resulting heat map will be incorrect.

Method 6 – Not Enough Data Points

This is usually the sign of a ‘lazy’ survey. Not only does the fact there is not enough data points lower the accuracy of the survey, more importantly it usually also comes with an increase in the Signal Propagation Assessment value to offset the lack of data to start with.

If the SPA is set to 5m, then there needs to be a red dot or a blue dot every 5m. Simple.  More dots is fine, but fewer dots than your SPA will result in white spaces showing up on the floor plan.  Then you might be tempted to increase the SPA in order to ‘hide’ these white spots.

Method 7 – Un-calibrated Survey

When you start a new site survey project, you are given the option to enter the drawing’s dimensions (not the building’s dimensions, but the drawing’s dimensions).  If you leave it blank, (normal) some survey applications, such as AirMagnet Survey, will calibrate your drawing to a square 120’ x 120’.  This will be your telltale sign you have NOT calibrated yet.

Calibration should be the first thing you do after starting a new project. If you forget, and go about your survey collecting data – it is very difficult, if not impossible to accurately adjust all the data to the new calibration after the fact.  So, if you see a 120’ x 120’ in the site dimensions you have a survey that is un-calibrated!

In AirMagnet Survey in particular, above the upper right corner of your drawing there is a little dimension line. This is NOT the dimension line of your building, but what AirMagnet is using. Just because you have one there does not mean your drawing is calibrated.

Also, if you ever see a floor plan that looks a bit ‘off’ – like the X or Y axis is skewed – this is also a telltale sign you have an un-calibrated drawing.

Method 8 – PDF or JPEG Only

As a customer, you should request the PDF report, possibly the survey graphics in JPG format, but most importantly are the actual survey files. This way you can do your own analysis.  

Another means of mis-representation is to not include the walking paths and data collection points.  This, coupled with a high SPA (Guess Range), can be a sign the presented survey data is far too optimistic.  

To properly analyze the survey data, you need to know where the data was collected and what data has been interpolated.

Don’t settle for just a report; require the data files along with the reports.

Method 9 – Showing Only the Results for a Single SSID

When analyzing survey data, it is very sometimes appropriate  to include ONLY a specific SSID in the data set. In fact, this is the preferred method when analyzing for a (STA’s) device’s specific design requirements.

But, when doing analysis on interference (collision domains), ALL RF signals and access points should be included.  These ‘other’ devices still have an effect on the collision domains of ‘your’ access points and clients (STAs).

The subset of collected data you are analyzing is important. Sometimes you need to look at ALL the data, and other times just the SSIDs you care about.

Conclusions

Whether it’s on purpose, or merely an oversight, the use of any of these aforementioned methods may present site survey data in an inaccurate light.

Be forewarned and prepared to make an accurate evaluation of the survey data presented to you.

If you are the one presenting Survey Data... be sure to take an accurate survey (following all seven rules) and don't use any of the above techniques. You don't want to be a 'cheater' do you?

Again another adage; Just because the software allows you to manipulate the presentation of the data doesn’t mean you should!

Additional Articles for Supporting WLAN Site Surveys
- 7 Rules for Accurate Site Surveys
- How to 'Cheat' On A Survey - Don't be a Victim
- How to Properly Analyze Survey Data
- The Fallacy of Channel Overlap
- Predictive Survey vs Onsite Survey - What's the Big Deal?
- How to 'Spec' your Network's Physical Layer
- Want, Don't Want, Don't Care - Meeting Design Specs
- The Truth about SNR - Where Did that 'N' Come From Anyway?
- What is an Access Point Anyway - Hub, Bridge, Switch or Router?
- Passive vs Active - What's All the Fuss About
- The False God of dB
- Meeting All Device Design Parameters

Blog Disclaimer: The opinions expressed within these blog posts are solely the author’s and do not reflect the opinions and beliefs of the Certitrek, CWNP or its affiliates.