关于GCU、沐曦GPGPU、MLU、0卡V100资源4月7日恢复上架的公告>>> 关于共建具身智能开源数据集的倡议>>> 关于云脑任务中统一路径访问方式的公告>>> 关于将启智集群GPU资源迁移至智算集群的公告>>>
PCL-Platform.Intelligence
/
PanGu-Alpha
Not watched
51
114
Fork 66
Code
Releases 0 Wiki Activity
Issues 7 Pull Requests 1 Datasets Cloudbrain
Issues
/
Detail
Labels Milestones
New Issue

#26 数据clean and filtering step 3: low-quality document filtering

Open
created 2 years ago by toothacher17 · 0 comments
toothacher17 commented 2 years ago

pangu-alpha团队您好，

在论文的3.1.1 Cleaning and filtering篇章中提到了训练了一个low-quality document filter following GPT3。但是在数据清洗repo（https://git.openi.org.cn/PCL-Platform.Intelligence/DataCollector） 里我看到了其他步骤用到的代码并且可以正确运行，但是这个low-quality document filter相关代码没有看到。请问这部分代码有公开计划吗？

多谢！

pangu-alpha团队您好， 在论文的3.1.1 Cleaning and filtering篇章中提到了训练了一个low-quality document filter following GPT3。但是在数据清洗repo（https://git.openi.org.cn/PCL-Platform.Intelligence/DataCollector） 里我看到了其他步骤用到的代码并且可以正确运行，但是这个low-quality document filter相关代码没有看到。请问这部分代码有公开计划吗？ 多谢！
Sign in to join this conversation.
No Branch/Tag Specified
Clear branch or tag
Branches Tags
Labels
Apply labels
Clear labels
No items
No Label
Milestone
Set milestone
Clear milestone
No items
No Milestone
Assignees
Assign users
Clear assignees
No Assignees
1 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Write Preview
Loading…
Cancel
Save
There is no content yet.
Delete Branch '%!s(MISSING)'

Deleting a branch is permanent. It CANNOT be undone. Continue?

No
Yes
Please read the following content carefully:

Dear OpenI User

Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.

For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》

Agree and continue
Disagree, exit