Parser Generator (三)

上篇提到了 parser generator 在产生程序码的时候，可以用四个步骤去拆解里面的资讯并产生程序码，我们现在来看一下范例吧！

进入范例之前，我们也复习一下之前提我们预计要产生出来的 class 内部程序码的结构大概是会长这个样子，我们就叫他 Code 1.1 。本篇文章有很多程序码都是要参考这个结构，对照着看才会比较有感觉。

Code 1.1

object RssItemParser {
	fun Element.getItem(): RssItem {
			// #1 Value Statement
			val titleTitle: String? = readString("title")
	    val titleItunesTitle: String? = readString("itunes:title")
	    val titleGoogleplayTitle: String? = readString("googleplay:title")
	    val authorAuthor: String? = readString("author")
	    val authorItunesAuthor: String? = readString("itunes:author")
	    val authorGoogleplayAuthor: String? = readString("googleplay:author")
	    val guidGuid: TestGuid? = getElementByTag("guid")?.getGuid()
	    val guidItunesGuid: TestGuid? = getElementByTag("itunes:guid")?.getGuid()
	    val guidGoogleplayGuid: TestGuid? = getElementByTag("googleplay:guid")?.getGuid()
	
			// #2 Use class constructor
      return RssItem(
	  		title = titleTitle ?: titleItunesTitle ?: titleGoogleplayTitle,
	  		author = authorAuthor ?: authorItunesAuthor ?: authorGoogleplayAuthor,
	  		guid = guidGuid ?: guidItunesGuid ?: guidGoogleplayGuid
	  	)
	}
}

接着就是呼叫四个步骤的程序码：

private fun getClassFunSpec(
        rootElement: Element,
        outputClassName: String,
        objectBuilder: TypeSpec.Builder
    ): FunSpec {
        val outputClass = ClassName(rootElement.getPackage(), outputClassName)
        val rssTag = rootElement.getAnnotation(RssTag::class.java)
        val rssRawData = rootElement.getAnnotation(RssRawData::class.java)
        if (rssTag != null && rssRawData != null) {
            logger.error("@RssTag and @RssRawData should not be used on the same class!", rootElement)
        }

        val tagName = rssTag?.name?.takeIfNotEmpty() ?: rootElement.simpleName.toString()
        rootTagName = tagName
        logger.log("[KotlinParserGenerator][getActionFunSpec] $rootTagName")
        val funSpec = FunSpec.builder(tagName.getFuncName())
            .receiver(elementClassName)
            .returns(outputClass)
        val propertyToParseData = mutableMapOf<String, ParseData>()
        topLevelCandidateOrder =
            rssTag?.order ?: arrayOf(OrderType.RSS_STANDARD, OrderType.ITUNES, OrderType.GOOGLE)

        val annotations = preProcessAnnotations(rootElement)

        rootElement.enclosedElements.forEach { preProcessParseData(it, propertyToParseData, annotations) }

        propertyToParseData.forEach { generateValueStatementForConstructor(it, funSpec, objectBuilder) }

        funSpec.addStatement("\nreturn $outputClassName(")
        // Generate constructor statements
        var index = 0
        val lastIndex = propertyToParseData.size - 1
        propertyToParseData.forEach {
            generateConstructor(it, funSpec, index == lastIndex)
            index ++
        }
        funSpec.addStatement("${TAB})")
        return funSpec.build()
    }

上面这段程序码包含了上述的四个步骤，分别可以对应到四个 function ：

1. Annotation Pre-processing： preProcessAnnotations
2. Convert to ParseData： preProcessParseData
3. Generate Value Statement： generateValueStatementForConstructor
4. Generate Constructor： generateConstructor

Annotation Pre-processing

首先，我们可以透过先把 annotation 里的 class 和 member 资讯整理好，放入 map 中以便之後快速查找，这边用 map 的原因是之後要取用 annotation 里面的资讯时，可以透过名称直接拿到资讯。

protected fun preProcessAnnotations(rootElement: Element): Map<String, Any?> {
        val result = mutableMapOf<String, Any?>()
        rootElement.enclosedElements.forEach { child ->
            if (child.kind != ElementKind.METHOD
                || !child.simpleName.isGetterMethod()
                || !child.simpleName.contains(ANNOTATION_SIGN)
            ) return@forEach

            val nameFromMethod = child.simpleName.extractNameFromMethod()

            val rssValue: RssValue? = child.getAnnotation(RssValue::class.java)
            val rssTag: RssTag? = child.getAnnotation(RssTag::class.java)
            val rssAttribute: RssAttribute? = child.getAnnotation(RssAttribute::class.java)
            val rssRawData: RssRawData? = child.getAnnotation(RssRawData::class.java)
            val nonNullCount = listOf<Any?>(rssTag, rssAttribute, rssRawData).count { it != null }
            val attributeValueCount = listOf<Any?>(rssAttribute, rssValue).count { it != null }
            if (nonNullCount > 1 || attributeValueCount > 1) {
                logger.error(
                    "You can't annotate more than one annotation at a field or property!",
                    child
                )
            }

            result[nameFromMethod] = rssValue ?: rssTag ?: rssAttribute ?: rssRawData

            if (rssValue != null) {
                hasRssValueAnnotation = true
            }
        }
        return result
    }

在处理 annotation 的时候，我们有给它一个限制就是不能在同一个元素上面标注超过一个 annotation ，所以才会有 if (nonNullCount > 1 || attributeValueCount > 1) 的判断。

Convert to ParseData

接着，我们就可以把这个 map 转成我们自定义的 ParseData 类别来包住一些我们需要的资讯：

data class ParseData(
    val type: String?,
    val rawType: String?,
    val dataType: DataType,
    val listItemType: String?,
    val packageName: String?,
    val processorElement: Element,
    val tagCandidates: List<String> = listOf()
)

protected fun preProcessParseData(
    child: Element,
    parseDataMap: MutableMap<String, ParseData>,
    nameToAnnotation: Map<String, Any?>
) {
    if (child.kind!= ElementKind.METHOD
|| !child.simpleName.isGetterMethod()
        || child.simpleName.contains(ANNOTATION_SIGN)
    ) return

    val nameFromMethod = child.simpleName.extractNameFromMethod()
    val exeElement = child as? ExecutableElement ?: return
    val rawType = exeElement.returnType.toString()
    val type: String?
    val packageName: String?
    val dataType: DataType
    var listItemType: String? = null

    if (rawType.isListType()) {
        listItemType = rawType.extractListType()
        type = rawType
            .substringBeforeLast('<')
            .extractType()
        packageName = rawType.substringAfterLast('<')
            .substringBeforeLast('>')
            .substringBeforeLast('.')
        dataType = DataType.LIST
} else {
        type = rawType.extractType()
        val annotation = nameToAnnotation[nameFromMethod]
        dataType = when {
            annotation is RssValue -> DataType.VALUE
annotation is RssAttribute -> DataType.ATTRIBUTE
rawType.isPrimitive() -> DataType.PRIMITIVE
else -> DataType.OTHER
}
        packageName = rawType.substringBeforeLast('.')
    }

    parseDataMap[nameFromMethod] = ParseData(
        type = type,
        rawType = rawType,
        dataType = dataType,
        listItemType = listItemType,
        packageName = packageName,
        processorElement = child,
        tagCandidates = getTagCandidates(nameToAnnotation, nameFromMethod, child)
    )
}

可以看到这边我们其实做了很多字串处理，为了要直接从 Element 的 rawType 中抽取出型别，型别抽取出来後把这些资料放到 ParseData 的 map 里面。

Generate Value Statement

利用上个步骤产生的 ParseData map ，我们可以开始把 map 里的资料转换成要产生 Code 1.1 注解 #1 内的程序码结构的... 程序码。（恩，很饶舌）

下面的程序码很长，但做的事情只有一件，就是把 ParseData 拆掉，按照 dataType 去产生 KotlinPoet 的 code statement 。

protected fun generateVariableStatement(
        propertyToParseData: Map.Entry<String, ParseData>,
        funSpec: FunSpec.Builder
    ) {
        val name = propertyToParseData.key
        val data = propertyToParseData.value
        val packageName = data.packageName ?: return
        val type = data.type ?: return

        when (data.dataType) {
            DataType.LIST -> {
                val itemType = data.listItemType ?: return

                if (itemType.isPrimitive()) {
                    val kClass = itemType.getKPrimitiveClass() ?: return

                    data.tagCandidates.forEach { tag ->
                        funSpec.addStatement(
                            "var ${name.getVariableName(tag)}: ArrayList<%T> = arrayListOf()",
                            kClass
                        )
                    }
                } else {
                    val itemClassName = ClassName(packageName, data.listItemType)
                    data.tagCandidates.forEach { tag ->
                        funSpec.addStatement(
                            "var ${name.getVariableName(tag)}: ArrayList<%T> = arrayListOf()",
                            itemClassName
                        )
                    }
                }
            }
            DataType.PRIMITIVE -> {
                val kClass = type.getKPrimitiveClass() ?: return

                data.tagCandidates.forEach { tag ->
                    funSpec.addStatement("var ${name.getVariableName(tag)}: %T? = null", kClass)
                }
            }
            DataType.ATTRIBUTE -> {
                val kClass = type.getKPrimitiveClass() ?: return

                data.tagCandidates.forEach { tag ->
                    val statement =
                        "var ${name.getVariableName(tag)}: %T? = getAttributeValue(null, \"$tag\")"
                            .appendTypeConversion(type)
                    if (type.isBooleanType()) {
                        funSpec.addStatement(statement, kClass, booleanConversionMemberName)
                    } else {
                        funSpec.addStatement(statement, kClass)
                    }
                }
            }
            DataType.VALUE -> {
                // Do nothing
            }
            else -> {
                val className = ClassName(packageName, type)
                data.tagCandidates.forEach { tag ->
                    funSpec.addStatement("var ${name.getVariableName(tag)}: %T? = null", className)
                }
            }
        }
    }

Generate Constructor

最後，我们可以利用前面已经产生的 Code 1.1 注解 #1 的区域，去产生 constructor ，也就是 Code 1.1 注解 #2 的区域，主要就是按照 dataType 去产生对应的 constructor 而已。

private fun generateValueStatementForConstructor(
        propertyToParseData: Map.Entry<String, ParseData>,
        funSpec: FunSpec.Builder,
        objectBuilder: TypeSpec.Builder
    ) {
        val name = propertyToParseData.key
        val data = propertyToParseData.value
        val packageName = data.packageName ?: return
        val type = data.type ?: return

        when (data.dataType) {
            DataType.LIST -> {
                generateValueListStatement(data, objectBuilder, packageName, funSpec, name)
            }
            DataType.PRIMITIVE -> {
                val kClass = type.getKPrimitiveClass() ?: return

                data.tagCandidates.forEach { tag ->
                    funSpec.addStatement("val ${name.getVariableName(tag)}: %T? =", kClass)
                    funSpec.addPrimitiveStatement("%M(\"$tag\")", type)
                }
            }
            DataType.ATTRIBUTE -> {
                generateValueAttributeStatement(type, data, name, funSpec)
            }
            DataType.VALUE -> {
                // Do nothing
            }
            else -> {
                val className = ClassName(packageName, type)
                data.tagCandidates.forEach { tag ->
                    val memberName = MemberName(type.getGeneratedClassPath(), tag.getFuncName())
                    funSpec.addStatement("val ${name.getVariableName(tag)}: %T? = %M(\"$tag\")?.%M()",
                        className, getElementByTagMemberName, memberName)
                }
            }
        }
    }

其他 Parser Generator 的产生方式

目前的范例都只有用纯 Kotlin 的 parser generator 去做讲解，其他像是 Android parser 和 reader generator 的程序码产生方式也是大同小异，看到这里你大概也有感受到，写 annotation processor 和 code generator 其实是一个苦工，你要从这些 annotation processor 给你的这些资讯，先过滤出你要的，然後用 KotlinPoet 去产生对应的程序码，而中间还会做一大堆的字串处理去拿到有用的资讯，真的是会很容易迷失在程序码里面。我自己写完这边的感受是，这个东西真的是写一次就好了... 可以解决很多重复程序码的问题，不用让大家跟我一起 “一袋米要扛几楼” 感受痛苦。

如果你也对其他的 parser generator 有兴趣，可以参考一下原始码：

• AndroidParserGenerator
• AndroidReaderGenerator
• ParserGenerator