使用Json.Net动态地从JSON中删除字段

Question

我有一些JSON输入，其形状我无法预测，并且我必须进行一些转换(调用它)，以便不记录某些字段。例如，如果我有这样的JSON:

{
    "id": 5,
    "name": "Peter",
    "password": "some pwd"
}

那么在转换之后，它应该是这样的:

{
    "id": 5,
    "name": "Peter"
}  

上面的例子是微不足道的，但实际情况并不那么容易。我将有一些正则表达式，如果任何字段(s)上的输入JSON匹配，那么它不应该在结果上。如果有嵌套的对象，我必须递归地处理。我已经看到了一些关于LINQ到JSON的东西，但是我发现没有什么能满足我的需求。

有办法做到这一点吗?

:这是日志库的一部分。如果有必要或更容易，我可以使用JSON字符串。问题是，在我的日志记录管道中的某个时刻，我获得了对象(或所需的字符串)，然后我需要从中剥离敏感数据，例如密码，以及任何其他客户端指定的数据。

您可以将JSON解析为JToken，然后使用递归助手方法将属性名称与正则表达式匹配。只要有匹配，你就可以从它的父对象中移除属性。在所有敏感信息被删除后，只需使用JToken.ToString()来获得编校的JSON。

helper方法可能是这样的:

public static string RemoveSensitiveProperties(string json, IEnumerable<Regex> regexes)
{
    JToken token = JToken.Parse(json);
    RemoveSensitiveProperties(token, regexes);
    return token.ToString();
}
public static void RemoveSensitiveProperties(JToken token, IEnumerable<Regex> regexes)
{
    if (token.Type == JTokenType.Object)
    {
        foreach (JProperty prop in token.Children<JProperty>().ToList())
        {
            bool removed = false;
            foreach (Regex regex in regexes)
            {
                if (regex.IsMatch(prop.Name))
                {
                    prop.Remove();
                    removed = true;
                    break;
                }
            }
            if (!removed)
            {
                RemoveSensitiveProperties(prop.Value, regexes);
            }
        }
    }
    else if (token.Type == JTokenType.Array)
    {
        foreach (JToken child in token.Children())
        {
            RemoveSensitiveProperties(child, regexes);
        }
    }
}

这里有一个简短的使用示例:

public static void Test()
{
    string json = @"
    {
      ""users"": [
        {
          ""id"": 5,
          ""name"": ""Peter Gibbons"",
          ""company"": ""Initech"",
          ""login"": ""pgibbons"",
          ""password"": ""Sup3rS3cr3tP@ssw0rd!"",
          ""financialDetails"": {
            ""creditCards"": [
              {
                ""vendor"": ""Viza"",
                ""cardNumber"": ""1000200030004000"",
                ""expDate"": ""2017-10-18"",
                ""securityCode"": 123,
                ""lastUse"": ""2016-10-15""
              },
              {
                ""vendor"": ""MasterCharge"",
                ""cardNumber"": ""1001200230034004"",
                ""expDate"": ""2018-05-21"",
                ""securityCode"": 789,
                ""lastUse"": ""2016-10-02""
              }
            ],
            ""bankAccounts"": [
              {
                ""accountType"": ""checking"",
                ""accountNumber"": ""12345678901"",
                ""financialInsitution"": ""1st Bank of USA"",
                ""routingNumber"": ""012345670""
              }
            ]
          },
          ""securityAnswers"":
          [
              ""Constantinople"",
              ""Goldfinkle"",
              ""Poppykosh"",
          ],
          ""interests"": ""Computer security, numbers and passwords""
        }
      ]
    }";
    Regex[] regexes = new Regex[]
    {
        new Regex("^.*password.*$", RegexOptions.IgnoreCase),
        new Regex("^.*number$", RegexOptions.IgnoreCase),
        new Regex("^expDate$", RegexOptions.IgnoreCase),
        new Regex("^security.*$", RegexOptions.IgnoreCase),
    };
    string redactedJson = RemoveSensitiveProperties(json, regexes);
    Console.WriteLine(redactedJson);
}

结果如下:

{
  "users": [
    {
      "id": 5,
      "name": "Peter Gibbons",
      "company": "Initech",
      "login": "pgibbons",
      "financialDetails": {
        "creditCards": [
          {
            "vendor": "Viza",
            "lastUse": "2016-10-15"
          },
          {
            "vendor": "MasterCharge",
            "lastUse": "2016-10-02"
          }
        ],
        "bankAccounts": [
          {
            "accountType": "checking",
            "financialInsitution": "1st Bank of USA"
          }
        ]
      },
      "interests": "Computer security, numbers and passwords"
    }
  ]
}

小提琴:https://dotnetfiddle.net/KcSuDt

您可以将JSON解析为JContainer(它是一个对象或数组)，然后使用DescendantsAndSelf()搜索JSON层次结构，以查找与某些Regex匹配的名称的属性，或与Regex匹配的字符串值，并使用JToken.Remove()删除这些项。

例如，给定以下JSON:

{
  "Items": [
    {
      "id": 5,
      "name": "Peter",
      "password": "some pwd"
    },
    {
      "id": 5,
      "name": "Peter",
      "password": "some pwd"
    }
  ],
  "RootPasswrd2": "some pwd",
  "SecretData": "This data is secret",
  "StringArray": [
    "I am public",
    "This is also secret"
  ]
}

您可以删除包含"pass.*w.*r.*d"的所有属性，如下所示:

var root = (JContainer)JToken.Parse(jsonString);
var nameRegex = new Regex(".*pass.*w.*r.*d.*", RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);
var query = root.DescendantsAndSelf()
    .OfType<JProperty>()
    .Where(p => nameRegex.IsMatch(p.Name));
query.RemoveFromLowestPossibleParents();

结果是:

{
  "Items": [
    {
      "id": 5,
      "name": "Peter"
    },
    {
      "id": 5,
      "name": "Peter"
    }
  ],
  "SecretData": "This data is secret",
  "StringArray": [
    "I am public",
    "This is also secret"
  ]
}

您可以删除包含子字符串secret的所有字符串值:

var valueRegex = new Regex(".*secret.*", RegexOptions.IgnoreCase);
var query2 = root.DescendantsAndSelf()
    .OfType<JValue>()
    .Where(v => v.Type == JTokenType.String && valueRegex.IsMatch((string)v));
query2.RemoveFromLowestPossibleParents();
var finalJsonString = root.ToString();

在第一个转换之后应用，结果是:

{
  "Items": [
    {
      "id": 5,
      "name": "Peter"
    },
    {
      "id": 5,
      "name": "Peter"
    }
  ],
  "StringArray": [
    "I am public"
  ]
}

为方便起见，我使用以下扩展方法:

public static partial class JsonExtensions
{
    public static TJToken RemoveFromLowestPossibleParent<TJToken>(this TJToken node) where TJToken : JToken
    {
        if (node == null)
            return null;
        JToken toRemove;
        var property = node.Parent as JProperty;
        if (property != null)
        {
            // Also detach the node from its immediate containing property -- Remove() does not do this even though it seems like it should
            toRemove = property;
            property.Value = null;
        }
        else
        {
            toRemove = node;
        }
        if (toRemove.Parent != null)
            toRemove.Remove();
        return node;
    }
    public static IEnumerable<TJToken> RemoveFromLowestPossibleParents<TJToken>(this IEnumerable<TJToken> nodes) where TJToken : JToken
    {
        var list = nodes.ToList();
        foreach (var node in list)
            node.RemoveFromLowestPossibleParent();
        return list;
    }
}