Avoid bot detection not working with Selenium

Question

I am accessing this website using a selenium chrome web driver, but whenever I submit the form it asks for me to verify that am not a bot. I have tried the following links (1, 2, 3) about avoiding detection but none of them worked.

     public void AvoidDetectionChrome(){
        System.setProperty("webdriver.chrome.driver", "path\\chromedriver.exe");

        //Set ChromeOptions
        ChromeOptions options = new ChromeOptions();
        options.addArguments("start-maximized");
        options.addArguments("--disable-blink-features=AutomationControlled");
        options.setExperimentalOption("excludeSwitches", new String[]{"enable-automation"});
        //Instantiate Web Driver
        WebDriver driver = new ChromeDriver(options);
        driver.get("https://www.americanairlines.ie/intl/ie/index.jsp");
    }

and code for Firefox driver:

public void AvoidDetectionFirefox(){
        System.setProperty("webdriver.gecko.driver", "path\\geckodriver.exe");

        //Set Firefox options
        FirefoxOptions options = new FirefoxOptions();
        options.addPreference("network.proxy.type", 1);
        options.addPreference("network.proxy.http", "localhost");
        options.addPreference("network.proxy.http_port", "3128");
        options.addPreference("dom.webdriver.enabled", false);
        options.setHeadless(false);

        //Instantiate Web Driver
        WebDriver driver = new FirefoxDriver(options);
        driver.manage().window().maximize();

        String url = "https://www.americanairlines.ie/intl/ie/index.jsp";
        driver.get(url);
    }

Both of them fail to avoid the bot detection and I get asked confirmation once I click the submit button, as could be seen in the image below: 

So how can I make any of the methods work when I submit the form and avoid the bot verification/detection?

Answer 1

Looks like the website is protected by Bot Management service provider Distil Networks and the navigation by ChromeDriver gets detected and subsequently blocked. Distil is like a bot firewall.

For more info refer to 

• Distil detects WebDriver driven Chrome Browsing Context
• Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection

Try removing the cdc_ portion from chromedriver.

For further understanding, you can refer to the Selenium Training.

Hope this helps!

Comments

Hi, thank you for your answer. The links you wrote and other threads like this suggest that I edit the chromedriver.exe and change variable name cdc_ to something else. But I have no idea how to do that or go about it?

---

All you have to do is open binary(chromedriver.exe) in hex editor, change string starting with $cdc to some other string of same length, then save and run modified binary.

---

Hi, I was able to edit it and it actually worked for a couple of requests and then started failing again. Using vpn to change my ip did not work either. Not sure what to do now. But thanks for the help and suggestions.

---

Try to maximize the browser window (rather than use a predefined size) it's detected less often. Hope this helps!