Crawling through multiple excel files match and copy data to master file

0 votes

I've created a macro that crawls through many excel files, matching and transferring the data into a master file. The excel files are all identical in terms of structure (columns, but row content may vary; there is a "key" though). But as the number of files increases, the time it takes for the macro to execute also increases. Perhaps someone has a more effective solution?

Sub DataCrawler()
 
    On Error GoTo HandleError
    Application.ScreenUpdating = False
    Dim objectFileSys As Object
    Dim objectGetFolder As Object
    Dim file As Object
    Set objectFileSys = CreateObject("Scripting.FileSystemObject")
    Set objectGetFolder = objectFileSys.GetFolder("pathName") ' location of folder with files
    Dim counter As Integer
    counter = 0
   
    ' macro opens one file after another and checks for every key, if data is available
   
    For Each file In objectGetFolder.Files
        Dim sourceFiles As Workbook
        Set sourceFiles = Workbooks.Open(file.Path, True, True)
       
        Dim lookUp As Range
        Dim searchRange As Range
       
        For i = 10 To 342 ' number of rows with key in master file
            Set lookUp = Cells(i, 31)
            Set searchRange = sourceFiles.Worksheets("tableName").Range("AE:AJ")
            ' if cell in master file related to the key is empty, copy data
            If IsEmpty(Cells(i, 35)) Then
                lookUp.Offset(0, 1).Value = Application.VLookup(lookUp, searchRange, 2, False) 
                lookUp.Offset(0, 2).Value = Application.VLookup(lookUp, searchRange, 3, False) 
                lookUp.Offset(0, 3).Value = Application.VLookup(lookUp, searchRange, 4, False)
                lookUp.Offset(0, 4).Value = Application.VLookup(lookUp, searchRange, 5, False) 
                lookUp.Offset(0, 5).Value = Application.VLookup(lookUp, searchRange, 6, False) 
            
            ' if cell in master file related to the key is already filled, skip
            Else
                
            End If
        Next
       
        sourceFiles.Close False
        Set sourceFiles = Nothing
    Next
HandleError:
    Application.EnableEvents = True
    Application.ScreenUpdating = True
 
End Sub
Jan 12, 2023 in Others by Kithuzzz
• 38,000 points
619 views

1 answer to this question.

0 votes

One application only. It would be quicker to use Match() to locate the row for the "key," then copy the information as an array, but it's hard to estimate how that would affect the total run time. That would depend on how many files you're opening and how well the procedure performed in that area.

Sub DataCrawler()
 
    Dim objectFileSys As Object, objectGetFolder As Object
    Dim file As Object, searchRange As Range, i As Long
    Dim m, wsData As Worksheet, wbSource As Workbook
    
    On Error GoTo HandleError
    Application.ScreenUpdating = False
    Application.EnableEvents = False
    
    Set wsData = ThisWorkbook.Sheets("Lookup") 'for example
    
    Set objectFileSys = CreateObject("Scripting.FileSystemObject")
    Set objectGetFolder = objectFileSys.GetFolder("pathName")
    
    For Each file In objectGetFolder.Files
        
        Set wbSource = Workbooks.Open(file.Path, True, True)
        Set searchRange = wbSource.Worksheets("tableName").Columns("AE")
        
        For i = 10 To 342 ' number of rows with key in master file
            If IsEmpty(wsData.Cells(i, 35)) Then
                m = Application.Match(wsData.Cells(i, 31).Value, searchRange, 0)
                If Not IsError(m) Then
                    wsData.Cells(i, 31).Resize(1, 5).Value = _
                       searchRange.Cells(m).Offset(0, 1).Resize(1, 5).Value
                End If
            End If
        Next
        wbSource.Close False
    Next file

HandleError:
    If Err.Number <> 0 Then MsgBox Err.Description
    Application.EnableEvents = True
    Application.ScreenUpdating = True
 
End Sub
answered Jan 12, 2023 by narikkadan
• 63,600 points

Related Questions In Others

0 votes
1 answer

How do I merge multiple excel files to a single excel file

You copy a worksheet from before each ...READ MORE

answered Dec 24, 2022 in Others by narikkadan
• 63,600 points
945 views
0 votes
1 answer

How to unmerge multiple cells and transpose each value into a new column in Pandas dataframe from excel file

Try this: df = pd.read_excel("Sample_File.xlsx", header=[0,1,2,3,4,5], index_col = ...READ MORE

answered Jan 8, 2023 in Others by narikkadan
• 63,600 points
2,584 views
0 votes
1 answer

Excel VBA search based on cell values into folders and sub-folders to get the file path and data

This will create a listing of all ...READ MORE

answered Jan 19, 2023 in Others by narikkadan
• 63,600 points
2,104 views
0 votes
1 answer

How to convert data from txt files to Excel files using python

Hi , there are few steps to ...READ MORE

answered Feb 16, 2022 in Others by Edureka
• 13,690 points
14,045 views
0 votes
0 answers

Convert Rows to Columns with values in Excel using custom format

1 I having a Excel sheet with 1 ...READ MORE

Feb 17, 2022 in Others by Edureka
• 13,690 points
994 views
0 votes
1 answer

Excel stock and sales data management

you must attach the event handler each ...READ MORE

answered Sep 23, 2022 in Others by narikkadan
• 63,600 points
793 views
0 votes
1 answer

Using Excel VLOOKUP() function across two sheets

The syntax for VLOOKUP is VLOOKUP(Lookup_Value,Table Array,Col_index_num,Range_lookup) OR, to start in ...READ MORE

answered Sep 30, 2022 in Others by narikkadan
• 63,600 points
1,004 views
0 votes
1 answer

Remove formulas from all worksheets in Excel using VBA

Try this : Option Explicit Sub test1() ...READ MORE

answered Oct 3, 2022 in Others by narikkadan
• 63,600 points
2,075 views
0 votes
1 answer

Uipath(RPA) : read data from the PDF file and write to Excel file

If you want to use UiPath and ...READ MORE

answered Oct 17, 2022 in Others by narikkadan
• 63,600 points
2,178 views
0 votes
1 answer
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP