{"expand":"renderedFields,names,schema,operations,editmeta,changelog,versionedRepresentations","id":"44784","self":"https://jira.geedge.net/rest/api/2/issue/44784","key":"OMPUB-1358","fields":{"issuetype":{"self":"https://jira.geedge.net/rest/api/2/issuetype/10004","id":"10004","description":"","iconUrl":"https://jira.geedge.net/secure/viewavatar?size=xsmall&avatarId=10303&avatarType=issuetype","name":"故障","subtask":false,"avatarId":10303},"components":[],"timespent":null,"timeoriginalestimate":null,"description":"*现象描述：*\r\n\r\n当地时间2024年7月8日03:30，YGN-FTR站点发生断电，持续至07:41恢复。随后出现”OLAP Yarn Server Down“告警，排查发现以下现象：\r\n * 相关节点服务器的重启日志中发现Yarn和Hdfs进程的频繁重启日志，无其他相关日志。\r\n * 使用jps命令检查时发现存在Yarn和Hdfs进程。 \r\n * 使用ps命令未发现Yarn和hdfs进程。\r\n * 执行守护脚本中的启动命令返回以下信息 datanode running as process 2994. Stop it first。","project":{"self":"https://jira.geedge.net/rest/api/2/project/10206","id":"10206","key":"OMPUB","name":"Operation and Maintenance","projectTypeKey":"business","avatarUrls":{"48x48":"https://jira.geedge.net/secure/projectavatar?pid=10206&avatarId=10715","24x24":"https://jira.geedge.net/secure/projectavatar?size=small&pid=10206&avatarId=10715","16x16":"https://jira.geedge.net/secure/projectavatar?size=xsmall&pid=10206&avatarId=10715","32x32":"https://jira.geedge.net/secure/projectavatar?size=medium&pid=10206&avatarId=10715"},"projectCategory":{"self":"https://jira.geedge.net/rest/api/2/projectCategory/10002","id":"10002","description":"系统运维","name":"MaintenanceDev"}},"fixVersions":[],"aggregatetimespent":null,"resolution":{"self":"https://jira.geedge.net/rest/api/2/resolution/10000","id":"10000","description":"该问题的工作流程已完成。","name":"完成"},"timetracking":{},"customfield_10401":null,"customfield_10104":null,"customfield_10402":null,"customfield_10105":"0|i05qvw:","customfield_10403":null,"customfield_10404":null,"attachment":[],"aggregatetimeestimate":null,"resolutiondate":"2024-07-08T18:08:55.425+0800","workratio":-1,"summary":"【M22】YGN-FTR站点断电重启后出现”OLAP Yarn Server Down“告警","lastViewed":null,"watches":{"self":"https://jira.geedge.net/rest/api/2/issue/OMPUB-1358/watchers","watchCount":1,"isWatching":false},"creator":{"self":"https://jira.geedge.net/rest/api/2/user?username=wangchengcheng","name":"wangchengcheng","key":"JIRAUSER10603","emailAddress":"wangchengcheng@geedgenetworks.com","avatarUrls":{"48x48":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=48","24x24":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=24","16x16":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=16","32x32":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=32"},"displayName":"王成成","active":true,"timeZone":"Asia/Shanghai"},"subtasks":[],"created":"2024-07-08T14:12:03.079+0800","reporter":{"self":"https://jira.geedge.net/rest/api/2/user?username=wangchengcheng","name":"wangchengcheng","key":"JIRAUSER10603","emailAddress":"wangchengcheng@geedgenetworks.com","avatarUrls":{"48x48":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=48","24x24":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=24","16x16":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=16","32x32":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=32"},"displayName":"王成成","active":true,"timeZone":"Asia/Shanghai"},"customfield_10000":"{summaryBean=com.atlassian.jira.plugin.devstatus.rest.SummaryBean@563f96f2[summary={pullrequest=com.atlassian.jira.plugin.devstatus.rest.SummaryItemBean@5fe0f32f[overall=PullRequestOverallBean{stateCount=0, state='OPEN', details=PullRequestOverallDetails{openCount=0, mergedCount=0, declinedCount=0}},byInstanceType={}], build=com.atlassian.jira.plugin.devstatus.rest.SummaryItemBean@776e5e0b[overall=com.atlassian.jira.plugin.devstatus.summary.beans.BuildOverallBean@2f4fa242[failedBuildCount=0,successfulBuildCount=0,unknownBuildCount=0,count=0,lastUpdated=<null>,lastUpdatedTimestamp=<null>],byInstanceType={}], review=com.atlassian.jira.plugin.devstatus.rest.SummaryItemBean@46b42949[overall=com.atlassian.jira.plugin.devstatus.summary.beans.ReviewsOverallBean@7d99cc32[stateCount=0,state=<null>,dueDate=<null>,overDue=false,count=0,lastUpdated=<null>,lastUpdatedTimestamp=<null>],byInstanceType={}], deployment-environment=com.atlassian.jira.plugin.devstatus.rest.SummaryItemBean@78024b8c[overall=com.atlassian.jira.plugin.devstatus.summary.beans.DeploymentOverallBean@2a0dbb7c[topEnvironments=[],showProjects=false,successfulCount=0,count=0,lastUpdated=<null>,lastUpdatedTimestamp=<null>],byInstanceType={}], repository=com.atlassian.jira.plugin.devstatus.rest.SummaryItemBean@4d943115[overall=com.atlassian.jira.plugin.devstatus.summary.beans.CommitOverallBean@7e5f8cc9[count=0,lastUpdated=<null>,lastUpdatedTimestamp=<null>],byInstanceType={}], branch=com.atlassian.jira.plugin.devstatus.rest.SummaryItemBean@14ebe6a5[overall=com.atlassian.jira.plugin.devstatus.summary.beans.BranchOverallBean@53fa691f[count=0,lastUpdated=<null>,lastUpdatedTimestamp=<null>],byInstanceType={}]},errors=[],configErrors=[]], devSummaryJson={\"cachedValue\":{\"errors\":[],\"configErrors\":[],\"summary\":{\"pullrequest\":{\"overall\":{\"count\":0,\"lastUpdated\":null,\"stateCount\":0,\"state\":\"OPEN\",\"details\":{\"openCount\":0,\"mergedCount\":0,\"declinedCount\":0,\"total\":0},\"open\":true},\"byInstanceType\":{}},\"build\":{\"overall\":{\"count\":0,\"lastUpdated\":null,\"failedBuildCount\":0,\"successfulBuildCount\":0,\"unknownBuildCount\":0},\"byInstanceType\":{}},\"review\":{\"overall\":{\"count\":0,\"lastUpdated\":null,\"stateCount\":0,\"state\":null,\"dueDate\":null,\"overDue\":false,\"completed\":false},\"byInstanceType\":{}},\"deployment-environment\":{\"overall\":{\"count\":0,\"lastUpdated\":null,\"topEnvironments\":[],\"showProjects\":false,\"successfulCount\":0},\"byInstanceType\":{}},\"repository\":{\"overall\":{\"count\":0,\"lastUpdated\":null},\"byInstanceType\":{}},\"branch\":{\"overall\":{\"count\":0,\"lastUpdated\":null},\"byInstanceType\":{}}}},\"isStale\":false}}","aggregateprogress":{"progress":0,"total":0},"customfield_10100":null,"priority":{"self":"https://jira.geedge.net/rest/api/2/priority/2","iconUrl":"https://jira.geedge.net/images/icons/priorities/high.svg","name":"High","id":"2"},"customfield_10200":null,"customfield_10400":null,"labels":["M22"],"environment":null,"timeestimate":null,"aggregatetimeoriginalestimate":null,"versions":[],"duedate":"2024-07-08","progress":{"progress":0,"total":0},"issuelinks":[],"comment":{"comments":[{"self":"https://jira.geedge.net/rest/api/2/issue/44784/comment/82983","id":"82983","author":{"self":"https://jira.geedge.net/rest/api/2/user?username=wangchengcheng","name":"wangchengcheng","key":"JIRAUSER10603","emailAddress":"wangchengcheng@geedgenetworks.com","avatarUrls":{"48x48":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=48","24x24":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=24","16x16":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=16","32x32":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=32"},"displayName":"王成成","active":true,"timeZone":"Asia/Shanghai"},"body":"告警原因：使用ps命令未发现Yarn和hdfs进程，相关进程已挂掉，由于pid文件存在，导致守护脚本无法启动进程。启动时会报错datanode running as process 2994. Stop it first。\r\n临时解决：删除pid相关文件\r\n后续解决：守护脚本(hadoop-2.7.1/sbin/dae-xxx.sh)在启动进程之前添加删除对应的pid文件(hadoop-2.7.1/pids/hadoop-root-xxx.pid)操作","updateAuthor":{"self":"https://jira.geedge.net/rest/api/2/user?username=wangchengcheng","name":"wangchengcheng","key":"JIRAUSER10603","emailAddress":"wangchengcheng@geedgenetworks.com","avatarUrls":{"48x48":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=48","24x24":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=24","16x16":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=16","32x32":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=32"},"displayName":"王成成","active":true,"timeZone":"Asia/Shanghai"},"created":"2024-07-08T16:48:45.977+0800","updated":"2024-07-08T16:48:45.977+0800"}],"maxResults":1,"total":1,"startAt":0},"votes":{"self":"https://jira.geedge.net/rest/api/2/issue/OMPUB-1358/votes","votes":0,"hasVoted":false},"worklog":{"startAt":0,"maxResults":20,"total":0,"worklogs":[]},"assignee":{"self":"https://jira.geedge.net/rest/api/2/user?username=wangchengcheng","name":"wangchengcheng","key":"JIRAUSER10603","emailAddress":"wangchengcheng@geedgenetworks.com","avatarUrls":{"48x48":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=48","24x24":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=24","16x16":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=16","32x32":"https://www.gravatar.com/avatar/9312bd750f38605f6a8d6425d20c8ad5?d=mm&s=32"},"displayName":"王成成","active":true,"timeZone":"Asia/Shanghai"},"updated":"2024-08-29T15:27:39.886+0800","status":{"self":"https://jira.geedge.net/rest/api/2/status/10103","description":"这一问题被认为是完成, 这项决议是正确的。问题已关闭可以重新开放。","iconUrl":"https://jira.geedge.net/images/icons/statuses/generic.png","name":"已关闭","id":"10103","statusCategory":{"self":"https://jira.geedge.net/rest/api/2/statuscategory/3","id":3,"key":"done","colorName":"green","name":"完成"}}}}