Apache Flink

Ah ok, I was confused, clearly it has an account on GitHub, now I get it

I mean lemme open the laptop to see if I can give you some more insights

I found your doc on confluence, but I don't think I have access to the entire list of VMs probably

image.png

Ah ok, here is why I cannot find the machine:

<@U03H7GR1QRW> if the above is correct, I can get access tomorrow morning and operate the troubleshooting procedure above :+1:

meaning that I need to get some permission through somebody in Ververica

Oh yea, I created this GCP project back in the day

with all the recent hacks you need to be careful who to trust :joy:

Great, looking forward to your message tomorrow

Damn you are right, but I inherently trust you as I always came to Flink Forward :flink:
I took the freedom to copy/paste screenshots but you are right :handshake:

I'm wondering whether the <https://github.com/flink-ci/pr-labeler|flink-ci/pr-labeler> is also affected. I'd expect that process needs extensive permissions as well to edit PRs? :thinking_face: The <https://github.com/flink-ci/git-repo-sync|flink-ci/git-repo-sync> isn't affected (<https://github.com/flink-ci/flink-mirror|flink-ci/flink-mirror> still gets updated; I assume because mirroring only requires read access to Flink's public repo).

ok, I updated <https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration> accordingly

I wonder if we should start running flinkbot at an ASF provided VM to avoid these outages

Do they provide VMs? The GHA migration should be another way to solve this issue. So, not sure whether we want to open that box with Apache Infra.

Indeed, GHA will solve this problem much nicher :wink:

Is <https://github.com/flink-ci/pr-labeler> still hosted on GCP, same machine? <@U03H7GR1QRW>

Wrote now to our internals, gonna ping you back once I got access and assessed the situation

the pr-labeler was hosted on a machine that was shared with one of the other two processes (cibot or repo-sync) as far as I remember

Yo, I am SSHed into the machine:

```2024-04-04 08:57:40,438 ERROR c.v.CiBot                 - GitHub server error.
org.kohsuke.github.GHFileNotFoundException: <https://api.github.com/repos/flink-ci/flink> {"message":"Bad credentials","documentation_url":"<https://docs.github.com/rest>"}
        at org.kohsuke.github.GitHubClient.interpretApiError(GitHubClient.java:480) ~[ci-bot.jar:?]
        at org.kohsuke.github.GitHubClient.sendRequest(GitHubClient.java:407) ~[ci-bot.jar:?]
        at org.kohsuke.github.GitHubClient.sendRequest(GitHubClient.java:355) ~[ci-bot.jar:?]
        at org.kohsuke.github.Requester.fetch(Requester.java:76) ~[ci-bot.jar:?]
        at org.kohsuke.github.GHRepository.read(GHRepository.java:131) ~[ci-bot.jar:?]
        at org.kohsuke.github.GitHub.getRepository(GitHub.java:625) ~[ci-bot.jar:?]
        at com.ververica.github.GithubActionsImpl.getBranches(GithubActionsImpl.java:189) ~[ci-bot.jar:?]
        at com.ververica.Core.getBranches(Core.java:221) ~[ci-bot.jar:?]
        at com.ververica.CiBot.tick(CiBot.java:191) ~[ci-bot.jar:?]
        at com.ververica.CiBot.runOnce(CiBot.java:158) [ci-bot.jar:?]
        at com.ververica.CiBot.run(CiBot.java:143) [ci-bot.jar:?]
        at com.ververica.CiBot.main(CiBot.java:123) [ci-bot.jar:?]
Caused by: java.io.FileNotFoundException: <https://api.github.com/repos/flink-ci/flink>
        at org.kohsuke.github.extras.okhttp3.ObsoleteUrlFactory$OkHttpURLConnection.getInputStream(ObsoleteUrlFactory.java:491) ~[ci-bot.jar:?]
        at org.kohsuke.github.extras.okhttp3.ObsoleteUrlFactory$DelegatingHttpsURLConnection.getInputStream(ObsoleteUrlFactory.java:1205) ~[ci-bot.jar:?]
        at org.kohsuke.github.GitHubHttpUrlConnectionClient$HttpURLConnectionResponseInfo.bodyStream(GitHubHttpUrlConnectionClient.java:196) ~[ci-bot.jar:?]
        at org.kohsuke.github.GitHubResponse$ResponseInfo.getBodyAsString(GitHubResponse.java:314) ~[ci-bot.jar:?]
        at org.kohsuke.github.GitHubResponse.parseBody(GitHubResponse.java:92) ~[ci-bot.jar:?]
        at org.kohsuke.github.Requester.lambda$fetch$1(Requester.java:76) ~[ci-bot.jar:?]
        at org.kohsuke.github.GitHubClient.createResponse(GitHubClient.java:449) ~[ci-bot.jar:?]
        at org.kohsuke.github.GitHubClient.sendRequest(GitHubClient.java:399) ~[ci-bot.jar:?]
        ... 10 more
2024-04-04 08:57:40,438 INFO  c.v.CiBot                 - Taking a nap...```

I don't see any pr-labeler here BTW, only ci-bot:

```chesnay@chesnay-cibot:/home/cibot$ ps -aux | grep java
chesnay   1743 11.4 35.3 2392508 615560 ?      Sl    2023 53923:43 java -XX:+HeapDumpOnOutOfMemoryError -Dlog4j.configurationFile=file:log4j2.properties -jar ./ci-bot.jar --interval 60 --observedRepository apache/flink --ciRepository flink-ci/flink --user flinkbot --githubToken ... --azureToken ... --backlog 24```

AFAIR, I documented the VMs in the ververica wiki. I'm not sure about the name of the VMs

I am on `chesnay-cibot` as specified in the docs :+1:

In the labeller, I get many lines similar to this:

```2023-05-11 20:09:45,361 WARN d.r.PullUpdater [check-pr-thread] Failed to extract Jira ID from PR '[DRAFT] Stubs for formats implementation for the new File Source (ORC, Avro, CSV)'.```

&gt; Get access to huddles with more than two people, and other <https://apache-flink.slack.com/plans|premium collaboration tools> when you upgrade to a paid plan.

<https://confluent.zoom.us/j/94076218466>

let's use google meet in case robert wants to join again

Curious thing, side note to ask to <@U03H7GR1QRW> :

There is a process:

```/usr/lib/jvm/java-8-openjdk-amd64/bin/java -classpath /usr/share/maven/boot/plexus-classworlds-2.x.jar -Dclassworlds.conf=/usr/share/maven/bin/m2.conf -Dmaven.home=/usr/share/maven -Dmaven.multiModuleProjectDirectory=/home/robert/production/flink-community-teams org.codehaus.plexus.classworlds.launcher.Launcher exec:java -Dexec.mainClass=de.robertmetzger.WebGenerator```
Logs:

```lorenzo_affetti@robert-flinkbot:/home/robert/production/flink-community-teams$ tail -f cron.log 
2024-04-04 09:42:06,300 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Querying JIRA for 'project = FLINK AND component IN ("API / State Processor", "Stateful Functions", "Build System / Stateful Functions")  AND createdDate &gt;= '2024-04-01 00:00' AND createdDate &lt; '2024-04-30 23:59''
2024-04-04 09:42:55,323 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Querying JIRA for 'project = FLINK AND component IN ("Table SQL / API", "Table SQL / Ecosystem", "Table SQL / Legacy Planner")  AND createdDate &lt;= '2024-04-30 23:59' AND (resolutiondate &gt; '2024-04-30 23:59' OR resolutiondate is EMPTY )'
2024-04-04 09:42:55,965 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Querying JIRA for 'project = FLINK AND component IN ("Table SQL / API", "Table SQL / Ecosystem", "Table SQL / Legacy Planner")  AND resolutiondate &gt;= '2024-04-01 00:00' AND resolutiondate &lt; '2024-04-30 23:59''
2024-04-04 09:42:56,052 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Querying JIRA for 'project = FLINK AND component IN ("Table SQL / API", "Table SQL / Ecosystem", "Table SQL / Legacy Planner")  AND createdDate &gt;= '2024-04-01 00:00' AND createdDate &lt; '2024-04-30 23:59''
2024-04-04 09:43:46,229 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Querying JIRA for 'project = FLINK AND component IN ("Table SQL / Planner", "Table SQL / Runtime", "Table SQL / API", "Table SQL / Client", "Table SQL / Ecosystem", "Table SQL / Legacy Planner")  AND createdDate &lt;= '2024-04-30 23:59' AND (resolutiondate &gt; '2024-04-30 23:59' OR resolutiondate is EMPTY )'
2024-04-04 09:43:46,880 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Querying JIRA for 'project = FLINK AND component IN ("Table SQL / Planner", "Table SQL / Runtime", "Table SQL / API", "Table SQL / Client", "Table SQL / Ecosystem", "Table SQL / Legacy Planner")  AND resolutiondate &gt;= '2024-04-01 00:00' AND resolutiondate &lt; '2024-04-30 23:59''
2024-04-04 09:43:46,976 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Querying JIRA for 'project = FLINK AND component IN ("Table SQL / Planner", "Table SQL / Runtime", "Table SQL / API", "Table SQL / Client", "Table SQL / Ecosystem", "Table SQL / Legacy Planner")  AND createdDate &gt;= '2024-04-01 00:00' AND createdDate &lt; '2024-04-30 23:59''
2024-04-04 09:44:34,734 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Querying JIRA for 'project = FLINK AND component IN ("Runtime / Task", "Runtime / Network")  AND createdDate &lt;= '2024-04-30 23:59' AND (resolutiondate &gt; '2024-04-30 23:59' OR resolutiondate is EMPTY )'
2024-04-04 09:44:35,359 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Querying JIRA for 'project = FLINK AND component IN ("Runtime / Task", "Runtime / Network")  AND resolutiondate &gt;= '2024-04-01 00:00' AND resolutiondate &lt; '2024-04-30 23:59''
2024-04-04 09:44:35,403 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Querying JIRA for 'project = FLINK AND component IN ("Runtime / Task", "Runtime / Network")  AND createdDate &gt;= '2024-04-01 00:00' AND createdDate &lt; '2024-04-30 23:59''
2024-04-04 09:45:26,544 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Querying JIRA for 'project = FLINK AND component IN ("Runtime / State Backends")  AND createdDate &lt;= '2024-04-30 23:59' AND (resolutiondate &gt; '2024-04-30 23:59' OR resolutiondate is EMPTY )'
2024-04-04 09:45:27,127 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Querying JIRA for 'project = FLINK AND component IN ("Runtime / State Backends")  AND resolutiondate &gt;= '2024-04-01 00:00' AND resolutiondate &lt; '2024-04-30 23:59''
2024-04-04 09:45:27,173 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Querying JIRA for 'project = FLINK AND component IN ("Runtime / State Backends")  AND createdDate &gt;= '2024-04-01 00:00' AND createdDate &lt; '2024-04-30 23:59''
2024-04-04 09:45:27,370 INFO d.r.DataComputer [de.robertmetzger.WebGenerator.main()] Skipping team with no components```
What does that do?

It renders the Flink community statistics

it is hosted in <http://community.data-artisans.com|community.data-artisans.com> or something

or <http://community.flink-project.org|community.flink-project.org>

I think I linked it into the wiki, some sort of "start page" in the wiki

the idea is that we can track how in the company has been active in the community with the mailing lists

and how the teams are doing wrt to open source work

Everything back to normality :+1:

Please, don't delete this channel as I might need some refresher when I update the internal docs :pray:

Unfortunately this morning I woke up quite sick, I need some rest now and cannot do now :sweat_smile:

Thank you guys for your cooperation :+1:
OSS brings people together :partying_face: :rolling_on_the_floor_laughing:

Only thing I am concerned about now is that the pr-labeler is apparently processing tons of PRs now.
It shares the token with the bot, so, it could be that it reaches GitHub's rate limit and compromises the functionality of the bot

but the rate limit is reset once per hour

is the limit per account or token? :thinking_face:

that's a good question. I suspect per account

I suspect the same. That would be too easy to work around the limit if it's per token. But ok, that means that it's ok for us to share the token per service