I am sorry that this Datacap Application doesn't follow the usual standards set with the Issue Template provided. I will provide answers to the Issue Template questions as accurate as i can and amend them with a bunch of Additional Information that might be relevant when considering my DataCap Application.
The original of this Datacap Application, written in markdown as a git repo README.md, can be found here, push mirrored to github here. I am not sure how well this will translate into a github issue and it might make sense for you to read the original README.md. Besides structural adjustments to accommodate github issues the content is the same.
- Note: github issues do not allow for internal linking. Links left as in the original README.md
[email protected] (source) will have had the chance to advise on structure and publication location (either https://github.com/fidlabs/Open-Data-Pathway or https://github.com/fidlabs/Enterprise-Data-Pathway) before hand - i am aware that this Datacap Application is out of the normal.
- initial mail send: Sat, 23 Nov 2024 10:35:29 +0100
- email exchange until Tue: 26 Nov 2024 10:21:19 +0100 - publish location chosen: https://github.com/fidlabs/Enterprise-Data-Pathway
I am not writing this lightheartedly, i am looking for a decision and not a discussion. I don't want to waste anyone's time. I will provide an Abstract of what this Datacap Application entails, followed by a Table of Contents to structure the latter part of this Datacap Application. The audience for this Datacap Application is the allocator this Datacap Application is directed at.
I am asking for 1.5PiB DataCap without answering at least one the 3 questions that normally need to be answered to get a DataCap allocation:
- Who we are
- How we are doing what we are doing
- What data we store
Additional Information on why i/we need DataCap and why i/we cannot answer these questions are added after the filled Large Dataset Notary Application template.
Version
2024-12-01T00:00:00.000Z
DataCap Applicant
Data Owner Name
unknown
Data Owner Country/Region
global, unknown
Data Owner Industry
unknown
Website
not available
Social Media Handle
not available
Social Media Type
Other
What is your role related to the dataset
facilitator, storage provider, owner
Total amount of DataCap being requested
1.5PiB
Expected size of single dataset (one copy)
1.5PiB, unknown
Number of replicas to store
1, unknown
Weekly allocation of DataCap requested
35TiB
On-chain address for first allocation
Will be provided when terms are agreed. Will be a new address
see Client Address
Data Type of Application
"No questions asked - No answers given" data
Identifier
If needed Filecoin team will fill up this field later on. You can skip this field.
Share a brief history of your project and organization
We store data for the better part of the last 2 decades. We do not ask questions and we do not give answers. Call it a hobby - we are far from being a "for profit" project. We store data on Filecoin, account for data we store on filecoin, for the last 3.5 years. We need DataDap to get a fair shot at being able to financially justify utilizing Filecoin for what we do
Is this project associated with other projects/ecosystem stakeholders?
unknown
If answered yes, what are the other projects/ecosystem stakeholders
Describe the data being stored onto Filecoin
unknown
Where was the data currently stored in this dataset sourced from
unknown
If you answered "Other" in the previous question, enter the details here
If you are a data preparer. What is your location (Country/Region)
global, unknown
If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?
If you are not preparing the data, who will prepare the data? (Provide name and business)
Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.
unknown
Please share a sample of the data
I can't and we won't. I do not have access to all of the data and it would be indistinguishable from /dev/random. We do not have the meta data to prove otherwise
Confirm that this is a public dataset that can be retrieved by anyone on the Network
I do not confirm
If you chose not to confirm, what was the reason
None of our SLAs contain default public retrieval
What is the expected retrieval frequency for this data
Varies
For how long do you plan to keep this dataset stored on Filecoin
Forever
In which geographies do you plan on making storage deals
Global, unknown
How will you be distributing your data to storage providers
Physical and digital media
How did you find your storage providers
Others
If you answered "Others" in the previous question, what is the tool or platform you used
The internet
Please list the provider IDs and location of the storage providers you will be working with.
Global, unknown
We will provide miner IDs as laid out here Miner IDs
How do you plan to make deals to your storage providers
Others/custom tool
If you answered "Others/custom tool" in the previous question, enter the details here
DDO, custom tooling
Can you confirm that you will follow the Fil+ guideline
I am not sure what the FIL+ guidelines are. I think we follow the principles laid out by https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0003.md
Well, it is how it is. We ponder this step for the last 2 years. If its necessary to give up information to be able to get a shot at running a Filecoin operation with a non ruinous cost profile, so be it. I think it is clear by now what information i cannot provide, i either do not have the information or handing it over would jeopardize what we are doing. This application will, if considered, produce some kind of information, most likely a Client Address and some Miner IDs, connecting off chain with on chain. I will not claim that we are happy about that.
We need money.
Our clients concern is data storage. We do not ask what data we store, we just store it. The dataset we are onboarding to Filecoin dates back to the early 2000s and evolves ever since. Its a diverse dataset, our SLAs are diverse - a lot of it is "forever" storage. We obviously know where the data comes from. Our main concern is disconnecting data source and data storage.
We are utilizing Filecoin for some of its unique features:
- the possibility to prove data storage with very limited data access needed
- being able to do so in a decentralized way, no direct communication between client and provider required
- the technology part scales practically effortless into the PiB range
But! Filecoin is expensive.
Our involvement with Filecoin dates back to pre mainnet times. In the terminology of Fil+: we are miners, clients, facilitators/prepares/..., its hard to draw a clear line. We are onboarding our data since there was reasonable tooling to do so. We do this, with some exceptions, constantly. Our operation is small and decentralized - somewhere between 1 and 15TiB sealing capacity per day, a lot of reutilizing hardware dedicated to other tasks. It evolved over time. We currently account for around half a dozen PiB on the Filecoin blockchain.
Around early 2022 it became more and more clear that we wouldn't be able to sustain a Filecoin operation without DataCap. As soon as GLIFs Defi Solutions became available on mainnet we started storing data for clients with DataCap allocations on separate miners. We never sought DataCap storage(deals) actively and just stored what came along the way - i am around a long time and a lot, you take what you can get, the math is too good. This helped us offsetting the overhead cost for sealing and storage until now. To be precise, until - roughly - mid 2025. Our main client needs to restructure and isn't able to acquire DataCap reliably to maintain the DataCap'ed storage of their data with us. Those DataCap'ed sectors will expire over the better part of 2025.
We do not want to lie. We need DataCap to store some our own data in a way that is cost effective to offset the part that isn't covered by DataCap. It is expensive to account for data storage on the Filecoin blockchain.
Please do the math yourself if you think i just want to steal your block rewards. This, 1.5PiB over the course of 2025, kinda results in 0.05% of overall DataCap live right now and - estimated, its hard to predict Fil+ allocation future - 0.5% of DataCap to be allocated over the course of 2025. At best. You know the numbers, i know them: 2026 will be a rough year, again. All i/we are asking for is a fighting chance.
I know most of you and most of you know me. I could have become a notary or an allocator, just made up clients or fired up the slack DM game to farm existing notaries/allocators with whatever reasoning. It doesn't benefit our use case! 4 years in i rather do this, butt naked here out in the open, than lie. I know i am asking for no strings attached, this could be as well a /dev/random scam, DataCap. It is not like we do not know the game. We were involved in onboarding and operation of upwards of 2EiB of sectors over the last 4 years - and we were involved in the off boarding upwards 2 EiB of sectors. We know the pain.
We are willing to give up our existing sealed sectors with DataCap - Existing DataCap Miners. We have no genuine interest (besides money) in extending sectors with DataCap that occupy storage space we could utilize better for our use case. We would seal all new sectors with the DataCap. The effective, new DataCap we would seal is max. 750TiB - if we can find the FIL for the pledge. The absurd situation we are in is that we are getting out marketed by DataCap for our own sealing power and storage.
We would store with other miners. We actually would love to. But Datacap prices us out of the market as a client. We would love to store backups of our data outside our own structures. But we do not want to apply for DataCap and can't pay 10$ per month. We don't need paper contracts, off chain payments, hot storage... we need to store data and get it back within the possibilities of the technology. If you run a boost and would just store data people transfer to you send us a message by opening non Fil+'ed deals on your miners, set prices you would consider for storage, unsealing and retrieval (again?!?, i know - we need to restart somewhere!). Be it for small parts of your storage you can afford to sacrifice for storage without DataCap. 17'ish miners will not be sufficient to provide any utility, we would need 100s. Feel free to find us on the network and send us your data - and FIL, sure. Nothing comes for free without DataCap... :/
Stability. Filecoin is having a rough one again this year and 2025 will not be better, i assume. Being able to plan at least a while ahead will take immense pressure of us in regards to how we conduct what we do and would give us a better chance of survival on our path into the future.
The raw numbers:
- 1.5PiB DataCap total
- we need around 35-50TiB/week
- over the course of 2025
We do not care how the DataCap is provided. We are flexible. We can do daily allocations if thats what it takes as long as its predictable and stable.
We have never:
- been a notary
- applied to be a notary
- been an allocator
- applied to be an allocator
We have applied for DataCap once. Surprisingly, for me, we got an allocation.
We did so to see how easy it is to get DataCap. We didn't intend to seal it with sectors and it just sat there untouched for a prolonged period of time. Eventually we sealed it with one of the DataCap Miners to test a workflow and it is apparently easier to get some amount of DataCap suitable for testing on mainnet than it is on the calibration network.
I will provide a client address without history. I/we have never done this with an unfunded address, if necessary we will fund the address to allow for datacap allocations.
I will provide 6 new miner IDs. All with the 10TiB storage power, 320 x 32GiB sectors, mining threshold acquired after terms for an eventual allocation are agreed on. All 6 miner IDs will eventually, when the pledge requires it, be owned by a single GLIF agent. The DataCap would be distributed equally to these miners IDs.
Everything that can be seen on chain we can provide in advance if necessary. Addresses, Data CIDs, DataCap Operations, Sealing Capacity, etc.
We can provide prove of ownership for our DataCap Miners miner IDs. We will let their sectors expire and will not extend any of the claims associated with them.
I choose this allocator to direct this Datacap Application at because of their special role in the ecosystem and especially Fil+ sphere. Filecoin Incentive Design Labs - FIDL is the only allocator operator we trust to not be a mining operation and, given what we are asking, we hope to get the stability we are seeking thru the special access - FIDL undoubtedly has - to the upper echelons of the FIL+ organizational team. It is out of the normal what i am asking - kinda, official approval and the stability that comes with it is what i am out for here.
I am not out for a discussion about FIL+. there was time, there will be time - this Datacap Application is not the place. I am looking to get a decision on this until the end of 2024 so we can adjust our operations appropriately, no matter the outcome. I have no intention to discuss this Datacap Application in DMs or any place private. Please keep it in the comments of this Datacap Application.
I am around a long time, i have done a lot especially in the earlier 2 years of the network. I am not looking for favours. I am looking for a decision if what i/we can provide in this application is enough to earn the trust to get a very limited, well contained amount of DataCap allocated to be able to keep on going what we are doing without being forced to store data only because it comes with DataCap.
If you'd consider granting us datacap please state so in the comments of this DataCap Application. I/we are not sure yet if i/we would consider pursuing other options to seek a DataCap allocation besides filing a DataCap request with this allocator.
Thanks for your time