Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
501 questions
0 answers
How to run a purge command in ADX using ADF?
Approach 1 Created an ADX activity in ADF, and in the command field, I wrote the purge command (which I copied from Azure Docs): .purge table MyTable in database DBName allrecords I tried running the purge command but it keeps returning syntax error:…
asked 2024-06-12T18:37:13.4033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 93%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESC%3C/text%3E%3C/svg%3E)
Sailesh Choyal
0
Reputation points
edited a comment 2024-06-25T13:36:25.0666667+00:00
Sailesh Choyal
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Cost-Effectiveness: ADF vs. Logic Apps for Azure Blob Storage
Good Afternoon, I am currently evaluating the cost-effectiveness of using Azure Data Factory (ADF) versus Azure Logic Apps for moving and deleting data in Azure Blob Storage. Could you please share your experiences and insights on the cost differences…
asked 2024-06-24T11:40:11.9333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 67%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFO%3C/text%3E%3C/svg%3E)
Femi Olunuga
20
Reputation points
accepted 2024-06-25T13:15:21.3433333+00:00
Femi Olunuga
20
Reputation points
0 answers
JIRA Tempo REST API with Copy Data Activity connection issue
Dear all, I would like to store the data from JIRA Tempo Account into my Azure Storage Account via Copy Data activity in Azure Synapse Workspace. It would be great if someone could help me on this. I create a new linked service with REST API and put…
asked 2024-06-25T10:28:48.1633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 75%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZM%3C/text%3E%3C/svg%3E)
Zhuoqiong Mo
20
Reputation points
commented 2024-06-25T13:09:14.4433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
Harishga
5,270
Reputation points Microsoft Vendor
0 answers
When I Ran the Pipeline I got Below error Could you please help out me?
Operation on target Execute pl_sap_dataload_full failed: Operation on target df_commonload_copy1 failed: The request failed with status code '"BadRequest"'.
asked 2024-06-25T13:06:35.13+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 12%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERC%3C/text%3E%3C/svg%3E)
Rajani Chikkala
0
Reputation points
asked 2024-06-25T13:06:35.13+00:00
Rajani Chikkala
0
Reputation points
0 answers
What could cause this error: Cluster creation failed due to an unexpected system error from a runtime related dependency
During one of the scheduled executions of a pipeline, an error occurred with the following details: Error code: [5000](https://go.microsoft.com/fwlink/?linkid=2117168#error-code-5000) Failure type: User configuration issue Details: Cluster creation…
Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
1,054 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,948 questions
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(96, 26%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV%3C/text%3E%3C/svg%3E)
1 answer
Azure Data Factory: Copy Data Activity: "Schema import failed: Please select a table"
Hello! I am pretty new to Azure Data Factory and trying to build my first data pipeline. So far, I have managed to install an integration run time in my PC and established a connection to the on prem SQL server. I have tested the connection and it says…
asked 2024-06-22T10:16:15.37+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 22%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EB%3C/text%3E%3C/svg%3E)
Bhargav
5
Reputation points
commented 2024-06-25T09:44:14.7333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
11,775
Reputation points Microsoft Vendor
1 answer
Can’t access storage in the sandbox
Hai I am trying to login to the sandbox environment in azure , not sure but not able to access to Azure storage account in the sandbox! Any help is appreciated
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 87%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDN%3C/text%3E%3C/svg%3E)
1 answer
Azure Data Factory pipeline failing on sink side: Job failed due to reason: at Sink 'sink1': Invalid object name '[Table_Name]'.
I have the following workflow: It fails on the sink step with: Job failed due to reason: at Sink 'sink1': Invalid object name '[table_name]'. The thing is that this table_name that shows up in the warning, I'm not using it nowhere in the whole data…
asked 2023-10-16T18:13:50.4333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 53%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETS%3C/text%3E%3C/svg%3E)
Thomaz Suzuki
5
Reputation points
commented 2024-06-25T09:26:32.06+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 57.00000000000001%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
Sudha Achuthan
20
Reputation points
1 answer
How to use wildcard for XML Files in Copy Data
I have a Azure Data Lake Storage Gen2 with following folder and file structure .\source .\source\SystemA_20240618.xml .\source\SystemB_20240618.xml .\source\SystemA_20240619.xml .\source\SystemB_20240619.xml I need to process only the files…
1 answer
How to transform sharepoint list items into an excel format
I'm trying to fetch the columns inside a sharepoint list using below MS Graph API:- sites/site-id/lists/list-id/items?expand=fields Below is the output that I'm getting:- { "@odata.context":…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 79%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
0 answers
How do I dynamically map columns in Data Factory to Dynamics 365?
Hi, I currently have a pipeline in Data Factory that will use a view from an Azure db to insert data into an entity in Dynamics 365. While this is working as expected, the next part of the process needs to update Dynamics 365. The number of fields that I…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 16%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDR%3C/text%3E%3C/svg%3E)
0 answers
Azure Data Factory - Jira Connector Schema Change / missing data
Hi All, We have been using ADF to ingest Jira into our data lake for several months now. The connector has worked with no issues and we have been able to pull all issues and all the custom fields, etc from Jira. Our Monday refresh failed due to a schema…
asked 2024-06-19T05:59:12.65+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 62%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
Jose
0
Reputation points
commented 2024-06-25T09:22:33.3066667+00:00
Smaran Thoomu
11,775
Reputation points Microsoft Vendor
1 answer
Data flow is not sinking all data because I have 84 json files in different sub-folders (of which 78 files are blank, and 6 files have data). But when I delete the 78 blank files (and remain with 6 files with data), the data flow will sink all results.
Hello. Please support, my data flow pipeline is not sinking all data. This happens whenever I have 84 json files each in different sub-folders (of which 78 files are blank, and 6 files have data). But when I delete the 78 blank json files (and only…
asked 2024-06-14T17:35:51.8166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 6%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Babu Musanga
0
Reputation points
commented 2024-06-25T09:20:02.8466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
7,595
Reputation points Microsoft Vendor
1 answer
Data Factory triggered from bob storage creating multiple runs
I have an ADF that currently triggers from a blob storage event, blob created, ignoring empty blobs. Previously this was correctly triggering a single ADF run when a new file was uploaded. I have recently introduced a Logic App which copies a new…
asked 2024-06-20T01:00:20.3033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 68%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELK%3C/text%3E%3C/svg%3E)
Louise Kirkham
0
Reputation points
commented 2024-06-25T09:19:17.4833333+00:00
phemanth
7,595
Reputation points Microsoft Vendor
1 answer
Base64 format of pdf file from Azure Data Factory
I am using Azure data factory to read a pdf file from a storage blob container and then convert it to base64 format using a Web activity. This is required to pass the base64 format to an API. Although the base64 value returned by the Web activity…
asked 2024-06-24T08:57:43.5733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(80, 80%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAP%3C/text%3E%3C/svg%3E)
Avinash Pai
0
Reputation points
commented 2024-06-25T09:18:10.1333333+00:00
phemanth
7,595
Reputation points Microsoft Vendor
1 answer
data factory data flow for SAP CDC connector
Hi all, I am working on getting CDC data from SAP using adf. Currently, I am facing issue to pass variables in using dynamic content dataflow. Refer to the diagram, it is working fine to pass variable in the Linked service properties for ODP context…
asked 2024-06-22T07:11:43.2766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 25%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYC%3C/text%3E%3C/svg%3E)
Yang Chow Mun
121
Reputation points
commented 2024-06-25T09:18:08.36+00:00
Smaran Thoomu
11,775
Reputation points Microsoft Vendor
0 answers
Azure Data Factory SFTP Linked Service: Failed to read binary packet data! (ProtocolError)
Hi everyone, I'm facing a problem with a SFTP server, where some restrictions are applied to avoid the usage of RSA Keys with SHA1 signature. I can connect without any problem to the SFTP server using WinSCP. There are no network restrictions so I can…
.NET
.NET
Microsoft Technologies based on the .NET software framework.
3,548 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,948 questions
asked 2024-06-17T09:23:48.5933333+00:00
Paul Hernandez
631
Reputation points Microsoft Employee
commented 2024-06-25T09:17:10.3933333+00:00
phemanth
7,595
Reputation points Microsoft Vendor
0 answers
Pipelines that require a scale up of the db are not scaling up
Hi, I have a couple of pipelines that have a stored procedure activity that scale up a database before the actual processing of the data starts. This has been working fine for almost a year, however in the past 2 days, the scale up times out and…
0 answers
Can we remove the artificial limitation of ADF Snowflake V2 supporting Multiple Statements
Issue with Snowflake V2 https://learn.microsoft.com/en-us/azure/data-factory/connector-snowflake?tabs=data-factory#differences-between-snowflake-and-snowflake-legacy How can I get feedback to the engineers that are working on this new connector for…
asked 2024-06-25T06:37:47.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 81%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMN%3C/text%3E%3C/svg%3E)
Macolino, Nick (Health)
1
Reputation point
commented 2024-06-25T07:39:21.3166667+00:00
Smaran Thoomu
11,775
Reputation points Microsoft Vendor
1 answer
Concept Ideas -> Big Table on Source where Rows are getting deleted and Target is DWH
Hi there, I need to have some idea related to "efficient data load". On source side, all tables has an primary key, created and updated timestamps. Therefor we have implemented on each pipeline an incremental load logic so that we are only…
asked 2024-06-25T06:41:41.8066667+00:00
Jörg Lang
100
Reputation points
commented 2024-06-25T07:33:52.2933333+00:00
Nandan Hegde
30,246
Reputation points MVP