Azure Data Explorer partitioning policy

0 votes
If I have a column with 100,000 cardinality. Shouldn't the maximum number of partitions be 100,000? Isn't it true that ADX should construct a partition for each different value in the column? Why is it even necessary to fill out the MaxPartitionCount property?
Mar 23, 2022 in Azure by Edureka
• 12,690 points
403 views

1 answer to this question.

0 votes
The end objective isn't to have a different partition for each individual value of the partition key in suggested cases (described in the doc you linked to).

Having a large number of partitions (100k in your case, or billions in the event of a unique device ID) can result in a large number of little data shards, which isn't ideal.

Even with "only" 128 partitions as the maximum, and default built-in indexing (regardless of explicit data partitioning), the ability to narrow the full data set down to a small number of partitions/shards at query planning time can result in significant resource utilisation and execution time savings.
answered Mar 25, 2022 by Edureka
• 13,620 points

Related Questions In Azure

0 votes
1 answer

Unzip gzip files in Azure Data factory

How to unzip a GZIP file? here's ...READ MORE

answered Mar 24, 2022 in Azure by Edureka
• 12,690 points
1,571 views
0 votes
0 answers

Testing & Deploy Azure Data Projects

As an Azure developer and data engineer, I enjoy automating cloud deployments and testing.  I'm currently working with a number of Azure resources, including SQL Server, PowerBI, Azure Data Factory, Azure Data Lake Storage, and Azure Data Lake Analytics. Doing some research I've found a bunch ...READ MORE

Feb 14, 2023 in Azure by Damonlang
• 1,230 points
223 views
0 votes
0 answers

Azure Data Engineer Associate certificate - preparation, Microsoft learn platform

Has anyone tried using Microsoft's online training ...READ MORE

Feb 14, 2023 in Azure by Damonlang
• 1,230 points
213 views
+1 vote
2 answers
0 votes
1 answer

What OS is Jenkins installed on Azure Virtual Machine?

Under Managed Jenkins there's a section called ...READ MORE

answered Apr 4, 2018 in DevOps & Agile by DareDev
• 6,890 points
555 views
+1 vote
3 answers

Where to find the Microsoft.IdentityModel dll?

Include Microsoft.IdentityModel.Extensions for WIF runtime support by following the following ...READ MORE

answered Oct 23, 2018 in Azure by Progbaaa
7,578 views
+1 vote
4 answers

How do Connect to Azure website via FTP?

First set up your FTP credentials are ...READ MORE

answered Oct 23, 2018 in Azure by abc
3,721 views
0 votes
1 answer

Azure Data Studio - Why Is Restore Option Missing Entirely?

The SERVERS sidebar allows you to access ...READ MORE

answered Mar 2, 2022 in Azure by Edureka
• 13,620 points
1,180 views
0 votes
1 answer

Can I create virtual machine without virtual network in Azure Resource Manager?

A VNet is used to provide the ...READ MORE

answered Mar 4, 2022 in Azure by Edureka
• 13,620 points
1,542 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP