如何删除json文件中的重复记录。 [英] how to remove duplicate records in a json file.

问题描述

我正在使用这样的joson文件读取数据

Hi,
I am reading data from joson file using like this

using (StreamReader r = new StreamReader(filename))
                {
                    string json = r.ReadToEnd();
                    dynamic array = JsonConvert.DeserializeObject(json);
                    foreach (var item in array)
                    {
                        Console.WriteLine("{0}", item.Title);
                        Console.WriteLine(item.Company);
                    }
                }

我的json文件包含这样的数据

my json file having data like this

{
		"Title": "Financial Analyst",
		"Company": "I Serve Systems Pvt Ltd",
		"Location": "Ganganagar",
		"Skill": "Gas, Oil, Energy, Financial Analysis, Finance",
		"Description": "The areas of Responsibility: * Work flow planning * Client liaison * Detailed ...",
		"Link": "http://jobsearch.naukri.com/job-listings-Financial-Analyst-I-Serve-Systems-Pvt-Ltd-Ganganagar-1-to-4-years-270914002180"
	}

{
		"Title": "Financial Analyst",
		"Company": "I Serve Systems Pvt Ltd",
		"Location": "Ganganagar",
		"Skill": "Gas, Oil, Energy, Financial Analysis, Finance",
		"Description": "The areas of Responsibility: * Work flow planning * Client liaison * Detailed ...",
		"Link": "http://jobsearch.naukri.com/job-listings-Financial-Analyst-I-Serve-Systems-Pvt-Ltd-Ganganagar-1-to-4-years-270914002180"
	}

以上两条记录是相同的，但我只想要一条记录。

我想要更高效的方式，因为我必须一次传递数千个文件。

请建议我。

谢谢。

Above two records are same, but I want only one record.
I want to more efficient way because I have to pass thousands of files at a time.
Please suggest me.
Thank you.

推荐答案

因为您使用Json.NET来处理JSON数据。最好为数据生成一个类，这将为您的数据提供基本模板。 JsonConvert会将JSON表示法转换为该类对象，并将类对象转换为JSON字符串表示法。



您的数据可以具有类对象模板作为

Since you're using Json.NET to work with JSON data. It will be better to have a class generated for the data, this will provide a basic template for your data to work on it. JsonConvert would convert the JSON notations into that class object, and will convert the class object into a JSON string notation.

Your data can have an class object template as

public class MyObject {
   public string Title { get; set; }
   public string Company { get; set; }
   public string Location { get; set; }
   public string Skill { get; set; }
   public string Description { get; set; }
   public string Link { get; set; }
}

这是您的数据模板，您可以将代码更改为此代码。

This is the template of your data, and you can change your code to this one instead.

// convert to MyObject
MyObject array = JsonConvert.DeserializeObject<myobject>(json);
// loop; remember, there is only one object here
// so i don't understand your loop logic
foreach (var item in array)
{
    Console.WriteLine("{0}", item.Title);
    Console.WriteLine(item.Company);
}

删除重复项



以上只是建议您使用。一旦你在列表中有多个项目，主要答案就是这个。单个项目始终是唯一的，重复发生在多个项目中。因此，首先将数据转换为列表。

Removing Duplicates

Above was just a suggestion for you to use. The main answer is this, once you're having multiple items inside your list. A single item would always be unique, duplication occurs in multiple items. So, first of all convert the data into a list.

// convert a list, add values
List<myobject> array = JsonConvert.DeserializeObject<list><myobject>>(json);
// get the distinct items..
// use the .ToList() to convert it back to a list.
array = array.Distinct().ToList();

以上代码首先将json转换为对象列表。然后它将仅选择不同的项目，之后它将该非重复列表保存到实际列表中。您可以使用Lambda表达式向Distinct方法添加条件。



您可以在此处了解有关Distinct方法的更多信息，http://msdn.microsoft.com/en-us/library/vstudio/bb920306(v=vs.90） .aspx [ ^ ]

如果你只是解析JSON来获取数据，也许你可以使用 json到csv转换器然后你可以使用Excel中的函数来删除重复项。

If you are just parsing the JSON to get at the data, perhaps you could just use a json to csv converter then you can just use the functions in Excel to remove the duplicates.

这篇关于如何删除json文件中的重复记录。的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！