Question

How can I capture network traffic of a specific page using Selenium

Answer 1

Hey, to capture network network traffic of a specific page using Selenium Webdriver:

public class CaptureNetworkTraffic {

public static WebDriver driver;
public static String driverPath = "C:\\\\Users\\\\Abha_Rathour\\\\Downloads\\\\ExtractedFiles\\\\chromedriver_win32\\\\chromedriver.exe";

@SuppressWarnings("deprecation")
public static void main(String[] args) {

System.setProperty("webdriver.chrome.driver", driverPath);

DesiredCapabilities caps = DesiredCapabilities.chrome();

LoggingPreferences logPrefs = new LoggingPreferences();
logPrefs.enable(LogType.PERFORMANCE, Level.INFO);

caps.setCapability(CapabilityType.LOGGING_PREFS, logPrefs);

driver= new ChromeDriver(caps);
driver.get("https://www.edureka.co/");

List<LogEntry> entries = driver.manage().logs().get(LogType.PERFORMANCE).getAll();

System.out.println(entries.size() + " " + LogType.PERFORMANCE + " log entries found");

for (LogEntry entry : entries) {
    System.out.println(new Date(entry.getTimestamp()) + " " + entry.getLevel() + " " + entry.getMessage());

}

driver.close();
driver.quit();

}
}

This will produce following output:

For further understanding, you can refer to the Selenium Course.

Comments

hi my friend i use your code but it doesnt work how should i do?

Exception in thread "main" org.openqa.selenium.InvalidArgumentException: invalid argument: log type 'performance' not found

  (Session info: chrome=76.0.3809.87)

Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'

System info: host: 'DESKTOP-BOLOAH3', ip: '168.158.152.158', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '1.8.0_212'

Driver info: org.openqa.selenium.chrome.ChromeDriver

Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 76.0.3809.87, chrome: {chromedriverVersion: 76.0.3809.126 (d80a294506b4..., userDataDir: C:\Users\Mobin\AppData\Loca...}, goog:chromeOptions: {debuggerAddress: localhost:1991}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: XP, platformName: XP, proxy: Proxy(), setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify}

Session ID: 31f02508307d36fbb666f0d4573cb6f2

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:187)

at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:122)

at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:49)

at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)

at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)

at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:552)

at org.openqa.selenium.remote.RemoteExecuteMethod.execute(RemoteExecuteMethod.java:35)

at org.openqa.selenium.remote.RemoteLogs.getRemoteEntries(RemoteLogs.java:81)

at org.openqa.selenium.remote.RemoteLogs.get(RemoteLogs.java:77)

at main.main(main.java:30)

---

Hi Mobin, did you use the exactly same code or you changed something in it.

---

Yeah Im Sure, Which Version of JDK and Selenium do you use ?

---

I am using Selenium 3.9.1 version and JDK of 11.0.2 version.

---

Hi! 

For anyone coming to this recently - in recent Selenium and ChromeDriver versions (eg 3.14.159, chrome driver 76.x) setting up loggingPrefs with ChromeDriver doesn't seem to work, using the local ChromeDriver - no matter what you do you seem to get the log type 'performance' not found error

The reason is increasingly strict w3c spec compliance, both in Selenium and ChromeDriver code.

Instead of the loggingPrefs/CapabilityType.LOGGING_PREFS capability, you need to use goog:loggingPrefs, like so

    ChromeOptions options = new ChromeOptions();
    LoggingPreferences logPrefs = new LoggingPreferences();
    logPrefs.enable( LogType.PERFORMANCE, Level.ALL );
    options.setCapability( "goog:loggingPrefs", logPrefs );

I guess that sooner or later the Selenium side will notice this and change the CapabilityType.LOGGING_PREFS constant or add a new one, but the project didn't look very friendly to log an issue with to me.

---

Hi,

Code works fine but log doesn't contain any information about ajax requests.Is there any possible way to include it? Thanks in advance.

---

Hello G3,

With this.driver.manage().logs().get(LogType.BROWSER).getAll() you should receive all logs that you can see in browser's console.