Testng XML not accepted by AWS Device Farm

0 votes

I'm trying to use the testng.xml... and looks like Device Farm is ignoring the whole file.

My example is simple. I have a factory class that instantiate test classes, and this is how my xml file looks like

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Default Suite">
    <test name="test">
        <classes>
            <class name="Factory"/>
        </classes>
    </test>
</suite>

I even tried to exclude methods manually

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Default Suite">
    <test name="test">
        <classes>
            <class name="Factory"/>
            <class name="TestSingleUrl">
                    <methods>
                        <exclude name="testCorrectSiteKey"/>
                        <exclude name="testIsTagImplemented"/>
                        <exclude name="testHttpsScheme"/>
                        <exclude name="testCorrectTagName"/>
                        <exclude name="testCorrectBootstrapPath"/>
                    </methods>
            </class>
        </classes>
    </test>
</suite>

But I'm getting a "Can't invoke method: either make it static or create a non-args constructor" . Which means that the Device Farm is trying to run the methods from TestSingleUrl Class..But the one who call these test should be the factory

Does anyone know how can we make the Device Farm to accept our xml file?

Oct 8, 2018 in AWS by eatcodesleeprepeat
• 4,710 points
1,785 views

This is my Factory Class:

public class Factory{

    private String targetSiteKey;

    //Data Provider that consumes the first line, which contains the target sitekey of the tag
    @DataProvider(name = "urlProvider")
    public Iterator<Object[]> createData() throws IOException {
        List<String> lines = IOUtils.readLines(new InputStreamReader(this.getClass().getResourceAsStream("Urls.csv")));
        List<Object[]> objectArrays = lines.stream().map(x -> new Object[]{x}).collect(Collectors.toList());

        Iterator<Object[]> itr = objectArrays.iterator();
        targetSiteKey = itr.next()[0].toString();

        return itr;
    }

    @Factory(dataProvider="urlProvider")
    public Object[] creatingTests(String url){
        System.out.println(System.currentTimeMillis());
        String siteKey=null;
        String path = null;
        String tagName = null;
        WebElement browsi;
        try{
            RemoteWebDriver driver = DriverWrapper.getDriver();
            driver.get(url);
            browsi = driver.findElement(By.id("browsi-tag"));

            tagName = browsi.getTagName();
            siteKey = browsi.getAttribute("data-sitekey");
            path = browsi.getAttribute("src");

        }catch(Exception ignored){}
        finally{
            return new Object[] {new TestSingleUrl(targetSiteKey,siteKey,path,tagName,url)};
        }
    }
}

And this is my TestSingleUrl Class:

public class TestSingleUrl {
    private String targetSiteKey,siteKey,path,tagName,url;

    public TestSingleUrl(String targetSiteKey,String siteKey, String path, String tagName,String url){
        this.targetSiteKey = targetSiteKey;
        this.siteKey=siteKey;
        this.path=path;
        this.tagName=tagName;
        this.url=url;
    }

    @Test
    public void testCorrectSiteKey(){
        System.out.println(url);
        Assert.assertEquals(siteKey, targetSiteKey);
    }

    @Test
    public void testIsTagImplemented(){
        System.out.println(url);
        Assert.assertFalse(siteKey.isEmpty());
        Assert.assertFalse(tagName.isEmpty());
        Assert.assertFalse(path.isEmpty());
    }

    @Test
    public void testHttpsScheme(){
        System.out.println(url);
        Assert.assertTrue(path.contains("https"));
    }

    @Test
    public void testCorrectTagName(){
        System.out.println(url);
        Assert.assertEquals(tagName,"script");
    }

    @Test
    public void testCorrectBootstrapPath(){
        System.out.println(url);
        Assert.assertTrue(path.contains("middycdn-a.akamaihd.net/bootstrap/bootstrap.js"));
    }
}

I think that the error messages I'm getting is because the device farm is trying to run every method with a @Test annotation that it finds. If it was reading my xml file it wouldn't happen.

1 answer to this question.

0 votes

I'm not sure exactly but I believe you should be able to place the testng.xml in a resources directory and reference it from the pom.xml .

Looks like this post shows how to reference the testng.xml from the pom.xml: How to call testng.xml file from pom.xml in Maven

When you build the project then it will be included in the jar.

The awslabs project builds it by using this mvn command

mvn clean package -DskipTests=true

https://github.com/awslabs/aws-device-farm-appium-tests-for-sample-app

[Update] I downloaded the sample project from awslabs and imported it into eclipse. I then added this plugin to the pom.xml file:

  <plugin>
           <groupId>org.apache.maven.plugins</groupId>
           <artifactId>maven-surefire-plugin</artifactId>
           <version>2.12.4</version>
           <configuration>
              <suiteXmlFiles>
                 <suiteXmlFile>src/test/resources/testng.xml</suiteXmlFile>
              </suiteXmlFiles>
           </configuration>
        </plugin>     

I then created the testng.xml in ./src/test/resources/

I copied and pasted the xml provided here:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Default Suite">
    <test name="test">
        <classes>
            <class name="Factory"/>
        </classes>
    </test>
</suite>

I then built the project using

mvn clean package -DskipTests=true

This produced a zip-with-dependencies.zip file in the target directory.

I uploaded this test package to device farm by creating a new project and selecting Appium Java Testng as the type of tests that I want to run.

I encountered this parse error:

Failed to generate the test suites[TestNG] [ERROR] Cannot find class in classpath: Factory

Which is expected since I don't have that class in my classpath, however, this proves that Device Farm did read the testng.xml.

[update]

I am able to reproduce the problem when I use the exclude tag in the testng.xml. I.E when I use this xml:

 <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Default Suite">
    <test name="test">
        <classes>
            <class name="factory.TestSingleUrl">
                <methods>
                    <exclude name="testCorrectSiteKey" />
                    <exclude name="testIsTagImplemented" />
                    <exclude name="testHttpsScheme" />
                    <exclude name="testCorrectTagName" />
                    <exclude name="testCorrectBootstrapPath" />
                </methods>
            </class>
        </classes>
    </test>
</suite>

Going to investigate further on the method tags.

[update]

Looks like this is a known issue in Device Farm: AWS Device farm seems to be ignoring TestNG annotations

Guess we will just need to wait for them to update their environment and the parser.

You might be able to run your tests on a private device. You'll need to contact aws to get one at this email:

aws-devicefarm-support@amazon.com

[Update]

Looks like they've updated the webDriverAgent. See if it works now.

Hope that helps.

answered Oct 8, 2018 by Priyaj
• 58,090 points

Related Questions In AWS

0 votes
1 answer
+3 votes
3 answers
0 votes
1 answer
0 votes
2 answers

What is the role of TestNG & JUnit frameworks in Selenium?

TestNG and JUnit are test frameworks . it ...READ MORE

answered Sep 4, 2020 in Selenium by Sri
• 3,190 points
2,486 views
0 votes
2 answers

Finding WebDriver element with Class Name in java

The better way to handle this element ...READ MORE

answered Apr 10, 2018 in Selenium by nsv999
• 5,500 points
12,617 views
0 votes
1 answer

Diff between WebDriver listeners and TestNG listeners?

This is not a very important concept ...READ MORE

answered Apr 14, 2018 in Selenium by king_kenny
• 3,710 points
6,105 views
0 votes
2 answers

Problem while using InternetExplorerDriver in Selenium WebDriver

enable trusted connection  in internet explorer by ...READ MORE

answered Aug 31, 2020 in Selenium by Sri
• 3,190 points
8,572 views
+1 vote
10 answers
+2 votes
3 answers
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP